Data Engineer

Position: Data Engineer w/Azure

Company Overview:

Direct Recruit Agency is a leading recruitment agency that specializes in providing top talent to companies across various industries. Our team is dedicated to connecting the best candidates with the right companies, and we are currently seeking a skilled Data Engineer to join our growing team.

Job Overview:

As a Data Engineer, you will be responsible for designing, building, and maintaining our company's data infrastructure. You will work closely with our data scientists, analysts, and other team members to ensure the accuracy, security, and accessibility of our data. Your role will be crucial in helping our company make data-driven decisions and achieve our business goals.

Key Responsibilities:

- Design and implement data pipelines to collect, process, and store large amounts of data from various sources.

- Develop and maintain data warehouses, data lakes, and other data storage solutions.

- Collaborate with data scientists and analysts to understand their data needs and provide them with the necessary tools and resources.

- Build and maintain data integration and ETL processes to ensure data accuracy and consistency.

- Monitor and troubleshoot data pipelines and systems to identify and resolve any issues.

- Implement data security measures to protect sensitive information.

- Stay updated on industry trends and best practices in data engineering and recommend improvements to our data infrastructure.

- Train and mentor junior data engineering team members.

Qualifications:

- Bachelor's degree in Computer Science, Information Systems, or a related field.

5+ years of experience in building out data pipelines in Python or Java
2+ years of experience working in Azure Cloud Service
Experience with data processing platform, such as Azure Data Factory
Experience with data lake/data marts/data warehouse
Experience with transactional database engines, such as SQL server
Fluent with SQL for data analysis
Excellent analytical and problem-solving skills with the ability to think quickly and offer
alternatives both independently and within teams.
Exposure working in an Agile environment with Scrum Master/Product owner and ability to
deliver

Experience with Spark
Experience working in Hadoop or other Big data platforms
Exposure to deploying code through pipeline
Good exposure to Containers like ECS or Docker
Working experience in a Linux based environment
Direct experience supporting multiple business units for foundational data work and sound
understanding of capital markets within Fixed Income
Knowledge of Jira, Confluence, SAFe development methodology & DevOps
Proven ability to work quickly in a dynamic environment.
Bachelor's degree Computer Science or a related field.

Ability to communicate the status and challenges with the team

Demonstrating the ability to learn new skills and work as a team

- Strong programming skills in languages such as SQL, Python, and Java.

- Experience with data warehousing and ETL tools such as AWS Redshift, Informatica, or Talend.

- Knowledge of data modeling and database design principles.

- Familiarity with cloud computing platforms such as AWS, Azure, or Google Cloud.

- Excellent problem-solving and analytical skills.

- Strong communication and collaboration skills.

Why Work for Us:

At Direct Recruit Agency, we value our employees and provide a supportive and inclusive work environment. We offer competitive salaries, benefits, and opportunities for career growth and development. As a Data Engineer, you will have the chance to work with cutting-edge technologies and make a significant impact on our company's success.

If you are a motivated and skilled Data Engineer looking for a challenging and rewarding opportunity, we encourage you to apply and join our dynamic team at Direct Recruit Agency.