ETL Developer - No C2C - Onsite Role - No Relo

Position Summary

We are seeking a highly skilled Senior Data Engineer to design, develop, and manage robust Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) pipelines entirely within the Amazon Web Services (AWS) ecosystem. This role is crucial for integrating diverse financial data sources into performant data warehouses and data lakes, ensuring scalability, security, and data quality. Deep domain experience within the Banking/Financial Services industry is mandatory.

Core Responsibilities

• Pipeline Architecture: Architect and implement robust, scalable ETL/ELT pipelines leveraging native AWS services for optimal data ingestion and processing.
• Data Integration: Integrate data from a variety of sources, including external APIs, transactional databases, and flat files, into centralized AWS-based data platforms.
• Transformation Development: Develop complex data transformation logic utilizing PySpark , Python , and SQL , primarily executed within AWS Glue and AWS Lambda environments.
• Orchestration & Monitoring: Establish, monitor, and maintain workflow orchestration using tools such as AWS Step Functions , Glue Workflows , or Apache Airflow on Amazon MWAA .
• Data Governance & Quality: Ensure stringent data quality, consistency, and lineage tracking, utilizing services like the AWS Glue Data Catalog and AWS Lake Formation .
• Performance Optimization: Proactively identify and execute optimizations for ETL performance and cost-efficiency through techniques like partitioning, parallelism, and resource tuning.
• Security & Compliance: Implement and enforce security best practices, including data encryption, management of IAM roles, and precise VPC configurations.
• Collaboration & Documentation: Partner closely with data engineers, analysts, and DevOps teams to support critical analytics and reporting needs. Maintain comprehensive documentation of ETL processes, data flows, and architectural diagrams.

Required Technical Stack

The candidate must demonstrate expert proficiency in developing scalable data solutions using the following AWS services and technologies:

• Serverless ETL: AWS Glue for serverless data integration and AWS Lambda for lightweight, real-time transformations.
• Data Storage: Amazon S3 for building and managing the core Data Lake.
• Data Warehousing: Experience with Amazon Redshift or Amazon RDS for relational data warehousing needs.
• Orchestration: Experience with AWS Step Functions, AWS Glue Workflows , or Apache Airflow on Amazon MWAA .
• Transformation Languages: Expert level skills in PySpark, Python , and SQL .