Utilizes software engineering principles to deploy and maintain fully automated data transformation pipelines that combine a large variety of storage and computation technologies to handle a distribution of data types and volumes in support of data architecture design.
Requirements
- Design, develop, and maintain data pipelines and ETL processes using Microsoft Azure services (e.g., Azure Data Factory, Azure Synapse, Azure Databricks, Azure Fabric).
- Utilize Azure data storage accounts for organizing and maintaining data pipeline outputs (e.g., Azure Data Lake Storage Gen 2 & Azure Blob storage).
- Collaborate with data scientists, data analysts, data architects and other stakeholders to understand data requirements and deliver high-quality data solutions.
- Optimize data pipelines in the Azure environment for performance, scalability, and reliability.
- Ensure data quality and integrity through data validation techniques and frameworks.
- Develop and maintain documentation for data processes, configurations, and best practices.
- Monitor and troubleshoot data pipeline issues to ensure timely resolution.
- Stay current with industry trends and emerging technologies to ensure our data solutions remain cutting-edge.
- Manage the CI/CD process for deploying and maintaining data solutions.
Benefits
- Competitive salary
- Health insurance
- Retirement plan
- Paid time off
- Professional development opportunities
To apply for this job please visit chevron.wd5.myworkdayjobs.com.
Follow us on social media