Senior DataOps/Cloud Data Engineer

Role Overview

We are seeking a Senior DataOps/Cloud Data Engineer to join our team. The successful candidate will have experience in data engineering, cloud data warehouse solutions, and data lakehouse solutions. The role will involve working with various data formats, cloud storage, and data processing orchestration.

What You Will Do

The Senior DataOps/Cloud Data Engineer will be responsible for designing, implementing, and deploying data engineering solutions using various tools and technologies. This will include working with AWS services, data lakehouse solutions, and data processing orchestration.

Why It Might Be a Fit

This role is a great fit for someone with experience in data engineering, cloud data warehouse solutions, and data lakehouse solutions. The successful candidate will have strong technical skills, excellent communication and problem-solving skills, and experience working in a public sector environment.

Requirements

• Data engineering (pandas, pyspark)
• Data lakehouse solutions such as Databricks (Delta Lake)
• Working with XLSX, CSV, JSON files, relational databases, cloud storage, structured and unstructured data
• Experience using AWS Services such as Glue, StepFunctions, Lambda, S3
• Extract/Transform/Load data using tools such as Informatica IDMC
• Programming & scripting: Python, SQL, Linux shell, PowerShell
• Data manipulation/analysis using pandas and pyspark
• Cloud experience: AWS and/or Azure services
• Cloud data warehouse solutions such as AWS Redshift
• Data processing orchestration/automation
• ETL tools (such as Azure Data Factory, AWS Glue) as well as cloud agnostic tools such as Informatica IDMC
• Data modeling & architecture: relational and dimensional modeling
• Data reporting/visualization
• Full SDLC from requirements gathering, design, implementation, testing to deployment and production support
• Project management (agile/scrum and waterfall)
• Change and Incident management
• Communication, presentation and negotiation skills
• Consulting, problem-solving and decision-making skills
• Public Sector experience