Data Engineer – Associate Director

The Databricks Data Architect – Associate Director will play a critical leadership role in designing, building, and scaling enterprise-grade data platforms using Databricks on Azure. This role combines deep hands-on expertise, solution architecture, and stakeholder engagement, making it ideal for a senior data engineering leader who can translate business objectives into scalable technical solutions while mentoring teams and driving best practices.

Requirements

• Architect, design, and deliver scalable data platforms using Azure Databricks, Apache Spark, and Delta Lake.
• Build and optimize end-to-end data ingestion, processing, and transformation pipelines for large, complex datasets from multiple sources.
• Develop reusable and standardized frameworks to improve performance, reliability, and maintainability of data engineering workloads.
• Lead and support cloud migration initiatives, including modernization of legacy data platforms to Databricks and Azure-native services.
• Implement data security, governance, and privacy controls, ensuring compliance with enterprise and regulatory standards.
• Monitor, troubleshoot, and performance-tune Databricks workloads with a focus on cost and compute efficiency.
• Collaborate closely with business stakeholders, analytics teams, and cross-functional technology teams to ensure data quality and availability.
• Provide technical thought leadership, including defining architecture standards, patterns, and Best Practices for Databricks adoption.
• Deliver proofs of concept (PoCs) and solution demos to showcase Databricks capabilities to stakeholders and clients.
• Mentor and guide data engineers, promoting high engineering standards and continuous improvement.
• Provide strategic and operational leadership for Data Engineering and Analytics teams to build data products, pipelines, and solutions that enable business intelligence, analytics, and AI/ML use cases.
• Oversee the development and deployment of scalable ETL/ELT pipelines, data marts, and analytics models that meet enterprise data needs.
• Ensure effective use of the generic frameworks, tools, and platforms to build, test, and deploy data solutions efficiently.
• Partner closely with peers to align on capabilities, enhancements, and optimizations needed for ingestion, transformation, orchestration, and deployment frameworks.
• Drive a data product mindset, ensuring that data assets are well-architected, high quality, governed, and reusable across the organization.
• Foster collaboration with Data Product Management, AI/ML teams, and business stakeholders to prioritize and deliver high-impact data solutions.
• Build and scale a high-performing team through recruitment, coaching, and mentoring, promoting a culture of innovation, inclusion, and continuous learning.
• Proven experience in leading large, distributed teams, managing organizational change, and scaling engineering functions.
• Lead transformational initiatives to modernize data architecture, adopt emerging technologies, and evolve engineering practices.
• Cultivate a collaborative and transparent culture, breaking down silos and aligning cross-functional teams around shared goals.
• Define clear objectives, KPIs, and performance metrics for teams, and provide feedback to ensure accountability and growth.
• Focus on automation, efficiency, and quality, while meeting or exceeding delivery timelines.
• Ensure adherence to data governance, privacy, and security policies in all data engineering and analytics work.
• Represent Data Engineering and Analytics in cross-functional forums, advocating for best practices, innovation, and alignment with enterprise architecture.

Benefits

• Generous Paid Time Off
• Parental Leave
• CSR Initiatives
• Networking Opportunities
• Employee Development Programs