Data Engineer

Role Overview

The Data Engineer will architect, design, and develop high-performance real-time and batch data pipelines that process massive volumes of data. They will work with modern data technologies to solve challenging engineering problems where scalability, performance, reliability, and data accuracy are essential.

What You Will Do

The Data Engineer will build data pipelines at scale, develop reliable data workflows, improve data quality and trust, build cloud-native data solutions, and develop scalable transformations. They will also own projects throughout the engineering lifecycle and collaborate closely with Data Science, Product, and Engineering teams.

Why It Might Be a Fit

The ideal candidate will have 3+ years of professional experience in software engineering or data engineering, with a focus on building data pipelines, distributed systems, or backend services. They will also have strong programming experience with Python, Java, and/or Scala, supported by solid computer science fundamentals.

Requirements

• 3+ years of professional experience in software engineering or data engineering
• B.S. or M.S. in Computer Science or a related technical discipline
• Strong programming experience with Python, Java, and/or Scala
• Strong understanding of data structures, algorithms, concurrency, and multithreaded programming
• Hands-on experience building distributed data processing systems using technologies such as Kafka, Spark, and Flink
• Experience developing and operating production data workloads using Databricks
• Experience designing and deploying solutions in AWS or a comparable cloud environment
• Experience building and managing data workflows using Apache Airflow
• Experience developing data transformations using dbt or similar frameworks
• Familiarity with data observability, quality, and reliability technologies such as Monte Carlo, Great Expectations, or equivalent tools

Tagged as: Python