Databricks Data Engineer

We are looking for a Databricks Data Engineer to help design, build, deploy, and maintain scalable and production grade data pipelines in modern cloud environments, enabling analytics, AI, ML, and decision advantage at scale. The ideal candidate will have a strong background in Databricks, PySpark notebooks, and cloud data engineering platforms, as well as experience with MLOps and machine learning.

Requirements

• Minimum of 3 years of experience in Databricks, PySpark notebooks, Python, DevOps, software development, and data engineering
• Certified Databricks Data Engineer Associate or Professional is a plus
• Proficient in designing, building, deploying, and maintaining high-performance, scalable ETL/ELT pipelines using Azure Databricks, Delta Lake, and PySpark Notebook
• Proficient in building, deploying, and operating production ML models such as supervised, unsupervised, and anomaly detection, including techniques for imbalanced datasets
• Proficient with ML engineering and MLOps, including model versioning, CI/CD for ML, monitoring, drift detection, and automated retraining
• Proficiency in Python including Pandas and PySpark Dataframes
• Expert level of SQL skills including Stored Procedure, experience with SSIS, SSRS, Power BI is a plus
• Proficient with cloud data engineering platforms, such as Azure, Databricks, Spark, or SQL, and batch and streaming pipelines
• Familiar with Databricks AI Built-In Functions such as AI_Query, AI_Gen, AI_Classify, AI_Forecast, AI_Analyze_Sentiment, able to use them to extract actionable insights from large amount of unstructured or structured raw data
• Experience with Python and ML frameworks, such as PyTorch or TensorFlow
• Experience improving data quality, lineage, and observability in enterprise data environments and operationalizing rules and model-driven scoring for prioritization, routing, or case selection
• Experience with predictive analytics, machine learning and artificial intelligence desired

Benefits

• Health
• dental
• vision
• life
• disability
• wellness
• paid time off
• 401(k)
• profit-sharing plans