The role
We are looking to hire a Senior Data Engineer to join our Data team in London.
This is an office-based role out of our London office.
Working at WGSN
Together, we create tomorrow
A career with WGSN is fast-paced, exciting and full of opportunities to grow and develop. We're a team of consumer and design trend forecasters, content creators, designers, data analysts, advisory consultants and much more, united by a common goal: to create tomorrow.
WGSN's trusted consumer and design forecasts power outstanding product design, enabling our customers to create a better future. Our services cover consumer insights, beauty, consumer tech, fashion, interiors, lifestyle, food and drink forecasting, data analytics and expert advisory. If you are an expert in your field, we want to hear from you.
Role overview
WGSN is expanding its AI & Data capability and strengthening its data foundation. As a Data Engineer, you will play a key role in building, optimising, and maintaining the data pipelines, models, and infrastructure that power our classification systems, AI workflows, forecasting models, TikTok insights, and consumer intelligence products.
You will work closely with senior data scientists, analysts, and engineers, particularly within the TikTok and Pulse pods, ensuring high-quality, well-modelled, reliable data flows across Snowflake, Databricks, and downstream systems. This role is ideal for someone with strong technical depth, advanced SQL and data modelling capabilities, and a passion for building scalable, efficient data systems.This is a hands-on, senior individual contributor role requiring at least 5+ years of experience in data engineering and the ability to mentor junior engineers when needed.
Key accountabilities
Data Architecture & Modelling
- Design, develop, and maintain scalable data architectures across Snowflake, Databricks, and cloud environments.
- Lead schema design, dimensional modelling, and query optimisation to support high-performance analytics and AI workloads.
- Collaborate with senior data scientists to structure data for classification, forecasting, embedding generation, and multimodal workflows.
Advanced SQL & Performance Optimisation
- Own complex SQL development and performance tuning across DS&E and DPS teams.
- Optimise costly queries, improve warehouse efficiency, and ensure best-practice SQL standards across shared codebases.
Pipeline Development (Batch & Near-Real-Time)
- Build robust ETL/ELT pipelines for ingestion, transformation, validation, and delivery.
- Develop resilient ingestion workflows for external APIs, including rate limiting, retries, schema drift handling, and monitoring
- (Future-facing) Support design of streaming or near-real-time data flows as product needs evolve.
Snowflake & Databricks Expertise
- Implement pipelines using Snowpark, PySpark, and distributed compute environments.
- Apply Snowflake performance optimisation, cost governance, RBAC, and Snowflake best practices.
- Support compute scaling across cloud platforms (AWS, GCP) and distributed cluster environments.
Data Quality, Contracts & Observability
- Implement data validation frameworks (e.g., Great Expectations) and enforce data contracts.
- Build monitoring, alerting, and lineage visibility for pipelines (e.g., dbt tests, metadata tracking).
- Ensure high standards of data accuracy, completeness, and reliability.
DataOps & CI/CD for Data
- Build automated CI/CD workflows for data using GitHub Actions, CircleCI, or similar.
- Develop automated unit tests, integration tests, and quality gates for data pipelines.
- Partner with DataOps & Platform Engineering to improve observability, documentation, and deployment workflows.
Workflow Orchestration
- Build and maintain orchestration workflows using Airflow, Prefect, Dagster, or equivalent.
- Optimise DAGs for performance, reliability, and clarity, while ensuring operational excellence.
Cloud Infrastructure, Containers & Runtime Management
- Run, log, monitor, and debug workloads across VMs, Docker containers, and cloud compute environments.
- Improve reliability and maintainability of containerised workloads powering AI and data pipelines.
Cross-Functional Collaboration
- Translate analytical and AI requirements into scalable engineering solutions.
- Document pipelines, decisions, runbooks, and architecture clearly and consistently.
Mentorship & Capability Building
- Provide guidance to junior engineers and contribute to building team-wide engineering maturity.
This list is not exhaustive and there may be other activities you are required to deliver.
Skills, experience & qualifications required
Experience
- 5+ years of hands-on experience as a Data Engineer.
- Proven success designing and scaling production-grade data pipelines in cloud environments.
- Experience mentoring junior engineers or contributing to capability uplift across teams.
Technical Skills
- Expert-level SQL: complex queries, optimisation, performance tuning, analytical SQL.
- Advanced data modelling (star schemas, normalisation, dimensional modelling).
- Strong Python skills, including Pandas, NumPy, and PySpark/Snowpark.
- Experience with Snowflake (performance optimisation, cost management, RBAC, governance).
- Experience with Databricks, distributed compute, and PySpark.
- Data pipeline orchestration (Airflow, Dagster, Prefect).
- Data validation frameworks (e.g., Great Expectations).
- Strong familiarity with cloud platforms (AWS or GCP).
- Experience building resilient API ingestion pipelines.
- Understanding of Docker, Linux servers, and cloud VMs.
- DataOps & DevOpsCI/CD workflows for data pipelines (GitHub Actions, CircleCI).
- Logging, monitoring, observability for data workflows.
Soft Skills
- Excellent communication across technical and non-technical teams.
- Ability to work within and contribute to cross-functional pods (DS + DE + Product + Content).
- Strong problem-solving skills and ownership mindset.