Data Engineer II

Role Overview

Design, develop, and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL processes to migrate and deploy data across systems.

What You Will Do

Design, develop, and maintain comprehensive data solutions, create efficient data pipelines, and ensure data integrity and quality throughout its lifecycle.

Why It Might Be a Fit

Must have skills in AWS Glue and PySpark, with strong experience in building and optimizing scalable data pipelines and workflows in cloud environments.

Requirements

• AWS Glue
• PySpark
• Strong experience in building and optimizing scalable data pipelines and workflows in cloud environments
• In-depth knowledge of data integration techniques and best practices for ETL processes
• Familiarity with data storage solutions and data lake architectures to support large-scale data processing
• Ability to troubleshoot and resolve complex data processing issues efficiently
• Experience with scripting and automation to enhance data pipeline reliability and performance