Data Engineer Databricks / Python / Database
Overview:
We are seeking an experienced Data Engineer with strong expertise in Databricks, Python, and Database engineering. The ideal candidate will design, build, and optimize scalable data pipelines, ensure data quality, and support advanced analytics and business intelligence initiatives across the organization.
- Design, build, and maintain ETL/ELT workflows on Databricks.
- Develop scalable data pipelines using Python and Spark.
- Optimize data transformation logic for performance and cost efficiency.
- Work with structured and unstructured data from various sources.
- Develop and manage data models, schemas, and tables across databases (SQL/NoSQL).
- Implement data quality, validation, and monitoring frameworks.
- Collaborate with Data Scientists, Analysts, and Cloud Engineers to support analytics and ML workloads.
- Ensure data security, governance, and compliance in all solutions.
- Troubleshoot pipeline failures and continuously improve pipeline reliability.
- Integrate data from APIs, streaming sources, and cloud storage.
Required Skills:
Core Technical Skills:-
- Databricks (mandatory) – Delta Lake, Notebooks, Spark SQL, Jobs, Workflows
- Python – Data processing, Spark APIs, automation, optimization
- Database Engineering – Strong SQL, query tuning, schema design, stored procedures
- Apache Spark – RDDs/DataFrames, optimisation, partitioning
- Cloud Platforms (one or more): AWS / Azure / GCP
- ETL/ELT tools: ADF, Glue, Dataflow, or equivalent
- Experience working with large-scale datasets
- CI/CD for data pipelines (Git, DevOps, Jenkins, etc.)
Recommended Jobs
PATIENT ACCESS SPECIALIST - FULL TIME - DAY
Patient Access Specialist - Full Time - Day JFK University Medical Center Edison, New Jersey Apply Requisition # 2026-175737 ShiftDay StatusFull Time with Benefits Overview Our tea…
Physician Hospitalist - Nocturnist
ChristianaCare , a nationally recognized leader in healthcare, is actively recruiting Internal Medicine, Family Medicine, and Internal Medicine-Pediatrics physicians to join its established Hospital…
MSL/Sr. MSL, Neurology - Alzheimer's Disease, Southern Ohio- Field based
At Eisai, satisfying unmet medical needs and increasing the benefits healthcare provides to patients, their families, and caregivers is Eisai's human health care (hhc) mission. We're a growing pharmac…
Show & Event Coordinator
Show & Event Coordinator Meteor Home Remodeling is one of the fastest growing brands in the acrylic bath remodeling industry. We are looking for quality people to join our winning team. Our goal …
Before Care Teacher for Preschool
Summary Start your day in a positive way! If you are looking for a part-time morning position that will have a lasting impact for students, we have the perfect position for you! The before care …
Electrical Engineer
NDI Engineering Company is looking for a dedicated and self-motivated entry/mid-level electrical engineer to assist in the design, implementation, maintenance, troubleshooting, and improvement of el…
Cashier
Coach is seeking a Cashier in Elizabeth, NJ, to manage customer transactions, maintain the checkout area, and provide outstanding service. The role requires strong attention to detail and proficiency …
Accounting/Human Resource Coordinator
Company: Empire Blended Products has been manufacturing specialty cement products since 1946. Our state-of-the-art, computerized manufacturing plant in Bayville, NJ produces a wide line of high-qual…