IN_Manager_AWS Data Engineer_Data and Analytics_Advisory_Bangalore

Job Description & Summary:  

A career within Data and Analytics services will provide you with the opportunity to help organisations uncover enterprise insights and drive business results using smarter data analytics. We focus on a collection of organisational technology capabilities, including business intelligence, data management, and data assurance that help our clients drive innovation, growth, and change within their organisations in order to keep up with the changing nature of customers and technology. We make impactful decisions by mixing mind and machine to leverage data, understand and navigate risk, and help our clients gain a competitive edge. 

Responsibilities: 

• Design, build, and maintain scalable data pipelines and ETL processes in AWS cloud environments. 

• Develop and manage file movement, data ingestion, and streaming solutions using AWS services such as Glue, Kinesis, Lambda, and Athena. 

• Implement and maintain data storage and processing solutions using AWS data services including Redshift, Aurora, RDS, S3, and related services. 

• Develop serverless and distributed data processing solutions using Python, Spark, and AWS Lambda. 

• Orchestrate workflows and pipelines using Apache Airflow and automation tools. 

• Implement CI/CD pipelines, version control, and automation using Jenkins, GitHub, and other DevOps platforms. 

• Ensure data security, access control, governance, and compliance by managing AWS security policies and access management. 

• Collaborate with cross-functional teams in an Agile environment to deliver reliable, high-quality data solutions. 

• Troubleshoot complex data issues and optimize performance of data workflows and systems. 

Mandatory skill sets: 

  Experience:  

• 6+ years of proven experience as a Data Engineer, preferably in cloud-based environments 

  AWS & Cloud Technologies:  

• AWS Glue, Athena, Lambda, Kinesis 

• AWS integration and ETL stack 

• AWS data services: Redshift, Aurora, RDS, S3 

• Security, IAM, policies, and access management in AWS 

  Programming & Data Processing:  

• Python 

• Apache Spark 

• AWS Lambda development 

  DevOps & Automation:  

• CI/CD pipelines 

• Jenkins 

• GitHub 

• Automation platforms 

  Workflow & Infrastructure Tools:  

• Apache Airflow 

• Ansible 

  Data Engineering Fundamentals:  

• Data modeling 

• Data warehousing concepts 

• Data architecture best practices 

  Methodologies:  

• Agile Development 

Preferred skill sets: 

• Experience with real-time data streaming architectures 

• Exposure to advanced AWS analytics services and performance optimization 

• Strong problem-solving and analytical abilities 

• Excellent communication and collaboration skills 

• Ability to work effectively in fast-paced, dynamic environments 

Years of experience required: 

6+ years of Data Engineering experience (Cloud/AWS preferred) 

Education qualification: 

BE, B.Tech, ME, M,Tech, MBA, MCA (60% above)