Data Engineer Operations, Assurant, GCC-India
Data Engineer reports to the Manager of Data Services and is responsible for developing Data Engineering solutions that conform to group standards, budgets and agreed estimated timelines. As the need arises, the Data Engineer will provide investigation for development and production data issues, tests and maintain scalable data solutions for structured and unstructured data to support reporting, analytics, ML and AI.
Data Engineer assists with optimization and the performance of bigdata eco-systems while assisting with research and building proof of concepts to test out theories recommended by Senior and Lead Data Engineers.
This position will be in Hyderabad/Bangalore/Chennai at our India location.
Work Timing:
12:30 PM - 9:30 PM IST (Regular)
3:30 PM IST- 12:30 PM IST (Once or Twice in a Week)
9:00 AM IST - 6:00 PM IST (Twice in a Year)
What will be my duties and responsibilities in this job?
- Responsible for the business intelligence requirements process, documents the requirements, coordinates deliverables from the BI resources, contributes to the technical design, deliverable of technical requirement documents, performs quality control and facilitates user acceptance testing for all BI deliverables.
- Responsible for data analytical and profiling functions.
- Liaison between the business and the IT resources.
- Gains working knowledge of related source data systems and tool sets.
- On smaller projects, the analyst coordinates the various resources, tasks and timing, and handles direct communication to end users.
- Assists with optimization the performance of bigdata ecosystems.
- Responsible for project intake management responsibilities such as project prioritization within the development teams, release/sprint allocations; and communication to business users, sponsors, and management.
- Established processes or methods are still relied on, however, you will be required to come up with creative solutions to problems.
- Constructs, tests and maintains scalable data solutions for structured and unstructured data to support reporting, analytics, ML and AI.
- Senior teammates are still needed to provide oversight on solutions to complex problems.
- Contributes to the development and Support of data-pipelines, and feature engineering of data solutions.
What are the requirements needed for this position?
- 5+ years of data warehousing experience using SQL Server BI Stack.
- 2+ year experience in Azure (Azure Data Factory, Azure Data Bricks).
- 2+ year experience in support team role.
- Strong written, verbal, and interpersonal communication skills
- Bachelor's Degree in Computer Science, Information Management Systems, or related subjects
What are the other requirements needed for this position?
- Knowledge of CRISP-DM methodology relevant to Data Engineering i.e: Data preparation and Deployment.
- Knowledge in Big Data technologies, concepts, and their applications for data processing.
- Advanced knowledge of Business Intelligence, Data Warehousing.
Knowledge in fundamentals of Machine Learning and Artificial Intelligence using Microsoft technologies. - Performance tuning and code optimization in SQL.
- Data profiling and dimension modeling techniques and creation of logical and physical data models.
- Experience working with job scheduling tools.
- Experience with SQL, C#, .NET, Python, Linux Shell Scripting or MS Power Shell, R or SAS, Spark, Hive, Pig, NoSQL
- Knowledge of Excel, MS word, MS PowerPoint
- Knowledge of relational database concepts and querying techniques using Microsoft SQL Server.
- Knowledge of requirements gathering practices.
- Understanding of Agile concepts.
- Understanding of basic Data-warehousing concepts.
- Great communication and time management skills.
- Great documentation skills.
- SQL scripting using MS SQL Server and Reporting using Power BI.