Data Engineer

Job Overview: 

As the Senior Manager for Data Engineering within the Data Enablement Competency, you will own the strategy, standards, and delivery of scalable data engineering capabilities across cloud and hybrid environments. You will lead multiple teams, set technical direction and best practices, and enable consistent, high-quality delivery through reusable frameworks, automation, and DataOps excellence. You will partner with senior stakeholders to shape roadmaps, oversee portfolios, manage risk and cost, and develop talent while fostering a culture of continuous improvement and Agile delivery. 

Key Responsibilities: 

Strategy, Architecture, and Governance 

• Define and evolve data engineering strategy, reference architectures, and standards for AWS, Azure, and on-premises environments (lakehouse, streaming, batch, ELT/ETL). 

• Establish and govern patterns for ingestion, transformation, orchestration, and storage (e.g., Spark/Databricks, Snowflake/Redshift/Synapse, Delta/Iceberg/Hudi). 

• Champion security, privacy-by-design, and compliance with data governance frameworks (e.g., Purview/Collibra), ensuring lineage, cataloging, and quality controls. 

• Drive FinOps and cost optimization across data platforms and workloads. 

DataOps and Platform Enablement 

• Own DataOps practices: trunk-based development or equivalent, Git workflows, automated testing, CI/CD, environment strategy, artifact management, and release governance. 

• Lead infrastructure as code practices (Terraform/Bicep/CloudFormation) and platform automation to enable repeatable, compliant deployments. 

• Implement observability and SRE principles: logging, metrics, tracing, SLIs/SLOs/SLAs, incident and problem management, and disaster recovery/BCP patterns. 

• Build and maintain reusable accelerators, templates, and guardrails to scale delivery across teams. 

Delivery Leadership and Portfolio Management 

• Oversee a portfolio of initiatives; manage roadmaps, capacity, risks, dependencies, and quality across multiple squads. 

• Ensure Agile at scale practices: backlog curation, sprint and release planning, OKRs/KPIs, and value tracking. 

• Collaborate with business and technology stakeholders to shape requirements, scope solutions, and align outcomes to business objectives. 

• Contribute to pre-sales, solutioning, estimation, and proposal development; support statements of work and delivery governance. 

People Leadership and Capability Uplift 

• Lead and develop high-performing teams (managers, senior associates, associates); drive hiring, coaching, performance management, and career progression. 

• Establish a community of practice; run training, code reviews, tech talks, and playbooks that uplift engineering maturity. 

• Promote an inclusive culture focused on learning, psychological safety, and continuous improvement. 

Operational Excellence 

• Ensure reliability, scalability, and performance of production pipelines and platforms; own major incident escalation and problem resolution. 

• Enforce quality assurance, secure coding, and data handling standards, including PII/PHI safeguards and regulatory adherence (e.g., POPIA/GDPR). 

• Track and report delivery and operational health to leadership; implement corrective actions and continuous improvement plans. 

Qualifications: 

• Bachelor's degree or Diploma in Computer Science, Information Systems, or a related field (like technology specific certifications). 

•  8+ years of hands-on data engineering experience with 3+ years leading teams and complex programs in cloud and hybrid environments. 

•  Demonstrable expertise across AWS and Azure data ecosystems: 

• Azure: ADLS, Data Factory, Synapse, Databricks, Purview 

• AWS: S3, Glue, EMR, Redshift, Lake Formation 

• Core: Spark, SQL, Python, orchestration (Airflow/Azure Data Factory), streaming (Kafka/Event Hubs), dbt, Snowflake/Delta 

• Strong DataOps background: Git-based workflows, CI/CD (Azure DevOps/GitHub/Jenkins), automated testing (unit/integration/data quality), artifact and environment management. 

• Infrastructure as code proficiency (Terraform preferred; Bicep/CloudFormation beneficial) and secure cloud foundations (IAM/RBAC, VNet/VPC, private endpoints, secrets management). 

• Proven track record in Agile delivery, portfolio management, and stakeholder engagement at senior levels. 

• Experience with data governance, cataloging, lineage, and data quality frameworks (e.g., Purview/Collibra, Great Expectations). 

• Excellent leadership, communication, and influencing skills; ability to navigate ambiguity and drive outcomes across cross-functional teams. 

Preferred/Advantageous: 

• Professional certifications (e.g., Azure Data Engineer/Architect, AWS Data Analytics/Architect, Databricks, Snowflake, Terraform). 

• Experience with SRE/observability tooling (Azure Monitor, CloudWatch, Datadog), containerization/orchestration (Docker/Kubernetes), and MLOps integration.