Data Engineer

Role Overview

The Data and Innovation section supports the agency to unlock the potential of our data to drive innovation and compliance for the agency. With a major uplift in data capability for the Agency, planned and delivered through our change program, there are vast opportunities to be a key contributor to drive, design and deliver policies, frameworks, and infrastructure to suit the agency’s data operation needs.

What You Will Do

Harmonizing the data migration, data architecture design and analytics across different schemes needed to effectively manage resources and delivery operations. Collecting data in an unformed and consistent way. Decrease the data migration complexity with well-defined common entity designs across the projects.

Why It Might Be a Fit

The role requires a strong understanding of data modelling, data structures, databases, and processes. Experience with processing large volumes of complex data and implementing Azure DecOps CI/CD with custom parameters for build and release pipelines for Data Factory, Synapse, and data lake artifacts is also required.

Requirements

• Minimum 7 years experience as a data engineer within the data and analytics domain
• Strong project delivery toolset experience in data integration tools
• Experience with processing large volumes of complex data
• Ability to assess complexity of data (volume, structure, relationship etc.)
• Strong understanding of data modelling, data structures, databases, and processes
• Implemented Azure DecOps CI/CD with custom parameters for build and release pipelines for Data Factory, Synapse, and data lake artifacts
• Hands on technical expertise implementing and delivering data solutions and pipelines in Azure
• Hands on technical expertise working with DataLake and Synapse Analytics
• Hands on technical expertise working with Dynamics 365 Dataverse and SQL data sources
• Hands on technical expertise setting up DevOps automation using tools such as GitHub etc.
• Experience building data profiling, data cleansing and data engineering frameworks
• Exceptional communication, documentation, and presentation skills