Data Engineer – II

As a Data Engineer, you will be involved in the entire development lifecycle, from brainstorming ideas to implementing scalable solutions that unlock data insights. You will collaborate with stakeholders to gather requirements, design data models, and build pipelines that support reporting, analytics, and exploratory analysis.

Requirements

• Design, build, and sustain efficient, scalable and performant Data Engineering Pipelines to ingest, sanitize, transform (ETL/ELT), and deliver high-volume, high-velocity data from diverse sources.
• Ensure reliable and consistent processing of versatile workloads of granularity such as Real Time, Near Real Time, Mini-batch, Batch and On-demand.
• Translate business requirements into technical specifications, design and implement solutions with clear documentation.
• Develop, optimize, and support production data workflows to ensure comprehensive and accurate datasets.
• With Software Engineering mindset/discipline, adopt best practices in writing modular code that is maintainable, scalable and performant.
• Use orchestration and scheduling tools (e.g., Airflow, GitLab Runners) to streamline workflows.
• Automate deployment and monitoring of data workflows using CI/CD best practices.
• Use DevOps best practices to instrument retries and self-healing to minimize, if not avoid manual intervention.
• Use AI/Gen AI tools and technologies to build/generate reusable code.
• Collaborate with Architects, Data scientists, BI engineers, Analysts, Product/ Program Mgmt and other stakeholders to deliver end-to-end solutions.
• Promote strategies to improve our data modelling, quality and architecture
• Mentor junior engineers, and contribute to team knowledge sharing.
• Have an analytics mindset to explore and identify opportunities for improved metrics, insights contributing to business growth.

Benefits

• Healthcare coverage
• Mental well-being support
• Retirement savings
• Paid time off
• Family leaves
• Complimentary games