Data Engineer & Insights Analyst

Role Overview

The Data Engineer & Insights Analyst will transform complex data into trusted, usable information to support program delivery, operational improvements, and executive decision-making in state government agencies. The role requires both technical depth and strong analytical judgment.

What You Will Do

Create and maintain optimal data pipeline architecture, design and implement cloud-based data solutions, build and maintain data orchestration workflows, and assemble large, complex data sets.

Why It Might Be a Fit

The right candidate will be excited by the prospect of optimizing the Commonwealth’s longitudinal data architecture to support our next generation of data initiatives.

Requirements

• Advanced SQL knowledge and proficiency working with relational databases, query authoring (SQL), as well as working familiarity with a variety of databases.
• Experience building and optimizing ‘big data’ data pipelines, architectures, and data sets.
• Advanced knowledge and proficiency designing, building, and refining dashboards, reports, analytical models, and decision-support products for executive and non-technical users.
• Experience working with student-level higher education data, including establishing linkages to other sources (e.g. employment and wage data) to measure outcomes and/or generate insights for decision-making).
• Experience performing root-cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
• Strong analytic skills related to working with unstructured datasets.
• Experience building processes supporting data transformation, data structures, metadata, dependency, and workload management.
• A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Hands-on experience with cloud-based data services such as AWS (e.g. S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, DynamoDB) (Required 5 Years)
• Proficiency in object-oriented/object function scripting languages such as Python, R, Java, C++, and Scala (Required 5 Years)
• Proficiency in relational SQL and NoSQL databases, including Oracle, MS SQL Server, PostgreSQL, and Cassandra (Required 5 Years)
• Experience with version control systems such as Git and CI/CD pipelines (Required 5 Years)
• Knowledge of data modeling techniques including Dimensional Modeling, Data Vault, and Star/Snowflake Schemas (Required 5 Years)
• Familiarity with data cataloging and metadata management tools such as AWS Glue Data Catalog (Required 5 Years)
• Understanding of data governance, data lineage, and data quality frameworks (Required 5 Years)
• Knowledge of REST API design and integration patterns (Required 5 Years)
• Experience with big data technologies such as Hadoop, Spark, and Kafka (Required 5 Years)
• Experience with data pipeline and workflow management tools such as Airflow, Azkaban, and Luigi (Required 5 Years)
• Experience with data integration solutions such as Informatica, MuleSoft, Talend, and TIBCO (Required 5 Years)
• Experience with stream-processing systems such as Storm and Spark Streaming (Required 5 Years)