What you do
Design, create and maintain data architectures that enable data extraction and transformation for predictive modeling
Design, implement, test, deploy, and maintain batch and stream data processing solutions
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources
Build cloud, scalable, real-time, and high-performance data lake solutions using Azure stack
Maintain data dictionary documentation on ML datasets
Create data tools for analytics and data scientist team members
What you offer
Degree in Computer Science or related fields
Advanced SQL proficiency
Knowledge of scripting languages (e.g. python, bash, shell)
Experience building and optimizing data pipelines, architectures and datasets
Experience with cloud platforms (MS Azure, AWS, Google Cloud)
Experience with data pipeline and workflow management tools (e.g. Data Factory, Informatica, SSIS, Pentaho…)
Design and development of ETL/ELT workflows
Familiarity with source control tools (e.g. Gitlab, Github, Bitbucket) and CI/CD process
Strong problem-solving skills
Plus points:
Knowledge of Azure services- Synapse, Data Factory, AML Studio, Azure Data Lake, Azure Databricks
Hands-on experience on big data frameworks like Spark, Kafka
Experience working on real-time data and streaming applications
Knowledge of python and python data packages (e.g. numpy, pandas…)
Machine learning concepts