Job Requisition ID #
Position Overview
Autodesk seeks a strong principal engineer and technical leader within the Data Acquisition team, designated with designing new robust and resilient data pipelines for analytical abstracts, enrichment, and metadata about design content. These datasets form the basis for data selection and understanding for various AI/ML projects at Autodesk.
This role collaborates with Product Managers, Program/Project managers, Analysts, Platform Architecture, and the Engineering Team to ensure data pipelines and delivery uses modern, scalable, reliable methods. Communication skills are critical, and you must be comfortable distilling complex technical information to a non-technical audience as a technical leader.
We work with a group of highly collaborative individuals who are excited to solve challenges, and implement cloud based, resilient, available, reliable solutions. We are comfortable with several languages and paradigms and aim to choose the technology which is most appropriate for the task. We endeavor to automate our tests and deployments end to end. We seek to mitigate existing issues and balance tech debt with the delivery of value to Autodesk. If you can identify with these traits, we are looking forward to working with you.
Responsibilities
Designing, architecting, solutioning, and implementing data pipelines for enterprise consumption
Mapping and automating data flows, from multiple sources
Develop workflows to run Apache Spark applications using Airflow and AWS EMR; may also work on Snowflake applications using Airflow and dbt
Deploy solutions via automated CI/CD processes to execute data pipelines running in batch
Developing, executing, and documenting software solutions to address complex data collection, processing, transformation, and reporting issues
Interpreting and translating business needs into technical requirements
Troubleshooting code level problems quickly and efficiently as needed
Participating in code and document reviews
Minimum Qualifications
Strong experience in several of the following: Pytho / PySpark, Spark / Spark SQL, Apache Iceberg/Airflow (Astronomer is a plus), Amazon EMR, Lambda, S3, Snowflake / Snowflake SQL, dbt, Fivetran, Hive, Hadoop
Extensive experience with a strong focus across layers of data engineering and pipelines
Strong experience in distributed and cloud architecture patterns
Ability to provide detailed architecture artifacts (L1 / L2 diagrams, sequence diagrams, etc.)
Firm understanding of programming paradigms, their usage, as well as data structure
Effective communication and collaboration skills, understanding how to tailor your message to your audience
Ability to fully develop a feature across all data and/or application layers from inception
Experience building and deploying data applications with AWS
Experience implementing CI/CD and familiarity TDD/BDD. Preferably concrete examples of each
Strong experience designing and using data lakes and data warehouses
Powerful sense of ownership with a bias for action when challenges arise
Ability to work independently and collaboratively across an organization.
Desire to create, investigate and follow best practices
Familiarity with AI/Machine Learning workflows and technologies is a plus
Eagerness to learn and share knowledge with a good attitude
Learn More
About Autodesk
Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.
We take great pride in our culture here at Autodesk – it’s at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.
When you’re an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future? Join us!
Salary transparency
Salary is one part of Autodesk’s competitive compensation package. Offers are based on the candidate’s experience and geographic location. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.Diversity & Belonging
We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging
Are you an existing contractor or consultant with Autodesk?
Please search for open jobs and apply internally (not on this external site).