Databricks Lead – Delivery & Engineering

Databricks Lead – Delivery & Engineering1

Lead hands-on Databricks delivery, ensuring adherence to CoE standards and best practices.

Key Responsibilities

• Lead Databricks engineering teams

• Build and optimize Spark & Delta Lake pipelines

• Implement Databricks Jobs, Workflows, CI/CD

• Optimize performance and DBU usage

• Contribute to Databricks accelerators and standards

• Design and implement end-to-end data platforms using Databricks Lakehouse

• Lead development of:

  • Data ingestion (batch & streaming)

  • Delta Lake data modeling (Bronze/Silver/Gold)

  • Scalable ETL/ELT pipelines

• Optimize Spark workloads for performance and cost

• Ensure reliability, scalability, and observability of data pipelines

Experience

• 10–12 years in data engineering

• 3–5 years hands-on Databricks experience

Skills

• Deep expertise in Databricks, Apache Spark, Delta Lake

• Strong hands-on experience with PySpark / Spark SQL  (core, SQL, streaming)

• Experience designing Lakehouse architectures

• Strong understanding of data modeling, data quality, and lineage

• Databricks platform features

• Cloud-native data engineering

• Performance tuning & troubleshooting

Certifications (Preferred)

• Databricks Data Engineer Associate and Professional