Data Architect
Job Title: Senior Architect – AWS | Kafka | | Glue Streaming; API consumption
Location : Dallas, Texas (Preferred) Or Florham Park, New Jersey , NJ
Full time
About the Role:
We are seeking a Senior Technical Lead /Architect with strong expertise in AWS-based streaming data pipelines, Apache Kafka (MSK), AWS Glue, Flink and PySpark, to help solution, design and implement a scalable data ingestion, data validation, data enrichment and reconciliation processing, and event logs, data observability, operational KPI tracking framework.
You will play a key role in solutioning and building out the , event driven capabilities with control gates in place to measure, track, and improve the operational SLAs and drive the data quality and reconciliation workflows for a high-impact data platform serving financial applications
Key Responsibilities:
Provide technical solution discovery effort on any new capabilities or new functionality.
Assist PO with technical user stories to ensure healthy backlog features
Lead the development of real-time data pipelines using AWS DMS, MSK, Kafka or Glue Streaming and for CDC ingestion from multiple SQL Server sources (RDS/on-prem).
Build and optimize streaming and batch data pipelines using AWS Glue (PySpark) to validate, transform, and normalize data to Iceberg and DynamoDB.
Define and enforce data quality, lineage, and reconciliation logic with support for both streaming and batch use cases.
Integrate with S3 Bronze/Silver layers and implement efficient schema evolution and partitioning strategies using Iceberg.
Collaborate with architects, analysts, and downstream application teams to design API and file-based egress layers.
Implement monitoring, logging, and event-based alerting using CloudWatch, SNS, and EventBridge.
Mentor junior developers and enforce best practices for modular, secure, and scalable data pipeline development.
Required Skills:
6+ years of hands-on expert level data engineering experience in cloud-based environments (AWS preferred) with event driven implementation
Strong experience with Apache Kafka / AWS MSK including topic design, partitioning, and Kafka Connect/Debezium
Proficiency in AWS Glue (PySpark) and for both batch and streaming ETL
Working knowledge of AWS DMS, S3, Lake Formation, DynamoDB, and Iceberg
Solid grasp of schema evolution, CDC patterns, and data reconciliation frameworks
Experience with infrastructure-as-code (CDK/Terraform) and DevOps practices (CI/CD,Git
Recommended Jobs
Electrical Maintenance Technician
Job Title: Maintenance Technician Job Description We are seeking a skilled Maintenance Technician / Maintenance Electrician responsible for ensuring the safe, reliable, and efficient operation …
Lead Backend Engineer- $2500 sign-on bonus ID48360
Join us and earn 2500 USD sign-on bonus AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank am…
Senior Director, Global Benefits
At Insmed, every moment and every patient counts - and so does every person who joins in. As a global biopharmaceutical company dedicated to transforming the lives of patients with serious and rare di…
Processing Operator
Foods you love. Brands you trust. And a career that empowers you to grow. At Nestlé USA, we’re all working towards the same goal – to delight and deliver for our consumers. With a rich portf…
Production Associate - Builder/Material Handler
Requisition Number: 219773 Job Description Cintas is seeking a Production Associate – Builder/Material Handler to support the Rental Division. The Builder/Material Handler is responsible for a…
Brand Ambassador
Kate Spade is seeking a Brand Ambassador in Blackwood, NJ, to represent the brand with professionalism and drive sales in a luxury fashion environment. The role involves engaging with customers, achie…
Home Health Aide
Home Health Aide (HHA) Caregiver Make a difference in someone's life! Join our team at Guardian Angel Senior services, where your compassion and dedication will improve the lives of our clients and…
Advanced or General Gastroenterologist at Hackensack Meridian Pascack Valley Medical Center in Westwood, NJ (28 mi from NYC)
Hackensack Meridian Pascack Valley Medical Center in Westwood, NJ, is seeking a BE/BC General or Advanced Gastroenterologist to join a supportive, well-integrated team offering a balanced mix of inpa…
Licensed Prosthetist / Orthotist (CPO)
We are actively recruiting a Licensed Prosthetist / Orthotist (CPO) to join our clinical team in Metuchen, NJ . This full-time opportunity offers a competitive annual salary range of $90,000–$125…
Sr. Rave Engineer - .Net
Job Title: Sr. Rave Engineer - .Net Category: Information Technology Description: Title: Medidata Rave Programmer Location: 100% REMOTE (Client is in EST) Type: Fulltime Rate/Sala…