Data Engineer

The Data Engineer plays a critical role in designing, building, and maintaining high-quality data pipelines on the Databricks platform. Success in the role requires writing efficient code, primarily in Python and SQL, while leveraging tools such as PySpark and Delta Lake.

Requirements

• Proficiency in data engineering principles, including the development and maintenance of data pipelines.
• Advanced coding skills in Python, SQL, and Scala, with significant experience working with Apache Spark.
• Hands-on experience with the Databricks platform, particularly with Delta Lake, Databricks Runtime, and Databricks Workflows.
• Familiarity with the Azure Cloud platform.
• Knowledge of the Gold Medallion architecture.
• Experience in data ingestion, transformation, and loading processes (ETL/ELT).
• Excellent communication skills, with the ability to explain complex data concepts to both technical and non-technical audiences.
• Strong problem-solving and analytical abilities.
• Experience with Mulesoft API platform is considered an asset.
• Background in creating ingestion pipelines from a variety of systems, such as HRIS, ERP, CRM, Microsoft SQL Server, and Apache Kafka.
• Experience with machine learning and data analytics.
• Knowledge of data governance and security best practices.
• Databricks certifications an asset
• Knowledge or experience in developing and integrating custom machine learning models using Azure Machine Learning, MLflow, and other relevant libraries

Benefits

• Paid time off
• 401(k) matching
• Retirement plan
• Health insurance