Senior Data Engineer
Hybrid commute, 2 days on-site in Central NJ Locations and 3 day remote per week. This is a permanent FT career opportunity, with base salary range 170 - 190K DOE, plus around 20% bonus and great benefits package. Looking for Senior Data Engineer in banking/financial industry. Major Responsibilities:
- Building and maintaining data pipelines to ingest data from source systems into Databricks
- Designing, Building and maintaining generic frameworks to ingest data through the medallion layers
- Working with the core data platform team to integrate the framework with reference architecture
- Assisting the data mesh teams with best practices and on-boarding to the new framework
- 10+ years of data engineering experience in building OLTP and OLAP databases, both on-prem and in the cloud
- 5+ years of experience with large scale data processing using SQL, Python, Pandas and Pyspark
- 5+ years of experience building robust and scalable data pipelines in Databricks Lakehouse platform on any key cloud platform provider, preferably Azure cloud
- Experience with implementing Medallion architecture for data processing
- Experience with implementing Streaming data ingestion using Kafka
- Test driven development experience
- Proven experience in solution design and applying best design practices
- Data modeling and ER/UML diagram skills
- Experience with orchestration/enterprise scheduler platforms like Airflow, Tidal etc..
- Experience with implementing API based data integration pipelines
- Experience with source control and DevOps for CI/CD
- Experience working in Agile/scrum methodologies
- Experience with Jira, confluence for project work and documentation
- Experience with Service now for incident management
- Finance domain knowledge in Investment Banking (Securities, Capital markets) and Security reference data knowledge
- Nice to have ADF and SSIS experience
- College degree required.
Check ALL our Jobs: Keywords: sql oracle python databricks etl data warehousing ssis ETL snow numpy pandas pyspark oltp olap
Recommended Jobs
Commercial Loan Servicing Specialist
At OceanFirst Bank, each one of our employees plays an important role in fulfilling the mission and taking responsibility to execute daily tasks in alignment with our core values. Together we foster a…
In-Home Sales Representative
Exterior Point is currently hiring in New Jersey for the role of in-home sales representative! We invest in you. We are a leading provider of high-quality exterior home improvement solut…
Machine Repair
Job Description Job Description Field service Technician Repair, Travel to local customers within 100 miles or less. Work is to be preformed at customer facilities in NJ,PA,NY,DE,CT. You will hav…
Head of CSX PMO Americas
What you´ll do Accountable for the Project Management Office within the CSX Americas Region. Responsible for ensuring successful delivery of key projects, large scale, and process optimization, al…
Unit Clerk / PCS / CNA / TSQ FED / FT / Days
Responsibilities Northwest Texas Healthcare System is a 495 bed system serving the city of Amarillo and the surrounding region. We offer the ideal combination of traditional values and the mo…
Associate Director - Wehrlen House
Are you passionate about making a difference in others' lives? Then come join the DPD team! Department for Persons with Disabilities (DPD) is one of the largest Catholic Charities Agencies providing …
Senior Accountant, Financial Reporting
Summary World Insurance Associates is a unique insurance organization offering top products and services from major providers, combined with attentive service from local agents. Founded in 2011…
Complex Staff Accountant- EnVue, Autograph Collection & Residence Inn, Weehawken, NJ
Complex Staff Accountant Location: EnVue, Autograph Collection & Residence Inn, Weehawken, NJ Department: Finance/Accounting Reports To: Director of Finance / Corporate Controller/General…
Client Success Coordinator
Description Position at WebMD WebMD is an Equal Opportunity/Affirmative Action employer and does not discriminate on the basis of race, ancestry, color, religion, sex, gender, age, marital status,…