Data Engineer

Architects and drives optimisation across enterprise-state of the art petabyte scale data pipelines and infrastructure foundational to the banks data driven strategy. Translates high volume, heterogeneous raw data into governed low latency and actionable data products for advanced analytics, machine learning and real time data insights that enable the delivery of accurate, high-quality, and timely data.

Requirements

• Architect, engineer and maintain robust scalable and fault tolerant data pipelines using technologies like Ab Initio, Python/Scala, Apache spark, Microsoft Fabric, Databricks for ingestion, transformation and delivery of structured, semi structured and unstructured data.
• Implement modern Data Ops practices including automation, testing and orchestration for both batch and low latency streaming data workflows.
• Collaborate with Software Engineers to design and implement consumption APIs and services that enable secure real time data access for business applications.
• Design and maintain the automated capture and maintenance of technical and operational metadata ensuring complete automated data lineage within the enterprise metadata hub (Ab Initio).
• Act as a technical guardian of the data integrity partnering with Data Governance to define and enforce master data management and data quality standards across all data products.
• Provide Level 2/3 support for complex data issues, coordinating with cross-functional teams to resolve incidents, ensuring a minimal mean time to resolution and clear communication
• Contribute to sprint planning, backlog refinement, and agile delivery processes.
• Drive continuous optimisation of data engineering practices, standards, and processes to improve efficiency and performance.
• Mentor and guide junior engineers, providing coaching and technical support to build capability.