Lead Data Engineer – Finance Tech

Role Overview

As a Lead Data Engineer, you will serve as the technical lead for the engineering team, designing and developing scalable, high-performance data solutions. You will drive the evolution of our data architecture, ensuring it meets both functional and non-functional business requirements while delivering reliability, efficiency, and scalability.

What You Will Do

Architect and build scalable, high-performance data pipelines and distributed data processing solutions using Spark, Scala/Java and cloud platforms (AWS, GCP, or Azure). Design and implement real-time and batch data processing solutions, ensuring data is efficiently processed and readily available for analytical and operational use.

Why It Might Be a Fit

You will play a key role in architecting and optimizing data pipelines using Hadoop, Spark, Scala/Java, and cloud technologies to support enterprise-wide data initiatives. You will collaborate with product teams and business stakeholders to understand requirements and deliver data-driven solutions.

Requirements

• BS or MS in Computer Science, Machine Learning, Artificial Intelligence, Applied Mathematics or a related technical field
• 7 plus years of experience in data engineering, software development, or distributed systems
• Expertise in Big Data technologies such as Hadoop, Spark, and distributed processing frameworks
• Strong programming skills in PySpark, Scala, and/or Java
• Experience with cloud platforms (AWS, GCP, or Azure) and associated data services (e.g. S3, BigQuery, Databricks, EMR, Snowflake)
• Proficiency in API development using REST, GraphQL, or gRPC to support real-time and batch data access
• Experience with real-time and streaming data architectures like Kafka, Flink or Kinesis
• Strong knowledge of data modeling, ETL pipeline design and performance optimization
• Understanding of data governance, security, and compliance in large-scale data environments
• Strong problem-solving skills and the ability to thrive in complex, evolving environments
• Excellent communication and collaboration skills, with experience working across cross-functional teams

Benefits

• Comprehensive health benefits and programs
• 401(k)
• Employee discount
• Short term disability
• Long term disability
• Paid sick leave
• Paid national holidays
• Paid vacation

Tagged as: BigQuery, Data Analysis, Snowflake