Role Overview
We are a AI-Native company operating in the transition to the post-routine business model. We work to help organizations and people rethink processes, workflows, and decision-making models through AI systems, agents, automation, and new collaborative modalities. We are looking for a hands-on figure with a broad knowledge of the Databricks platform, capable of operating in enterprise environments and enabling workload analytics and AI.
What You Will Do
Design and industrialize data solutions on Azure Databricks, overseeing the entire lifecycle: Lakehouse architecture, ingestion, transformation, orchestration, governance, data serving, and production.
Why It Might Be a Fit
A senior Data Engineer who knows Azure Databricks 360°: not just PySpark and pipelines, but also Lakehouse architecture, orchestration, Unity Catalog, SQL, governance, monitoring, costs, and integration with the rest of the data and AI ecosystem.
Requirements
- Proven experience on Azure Databricks in production, with end-to-end knowledge of the platform: workspace, environments, compute, cluster, and serverless configurations, and integrations.
- Practical knowledge of the Lakeflow stack (Connect, Jobs, Spark Declarative Pipelines), Auto Loader, and ingestion and orchestration modalities.
- Solid knowledge of Unity Catalog: catalogs, schema, privileges, lineage, external locations, access, and governance principles, security, and sharing.
- Experience with Databricks SQL and SQL Warehouses, data serving, Spark UI, query monitoring, and troubleshooting of the platform.
- Excellent mastery of Python, PySpark, SQL, and distributed data processing with Apache Spark, with experience in job optimization.
- Experience with Delta Lake: ACID tables, MERGE, CDC, schema evolution, and efficient management of incremental data.
- Ability to manage CDC, idempotence, deduplication, data validation, backfill, and data quality.
- Knowledge of Azure Data Lake Storage Gen2, Microsoft Entra ID, application identities, secret management, storage access, and enterprise networking.
- Ability to prepare environments, security, and permissions in collaboration with IT architectures, platform, security, and data governance teams of the client.
- Git, code review, Python packaging, automated testing, and CI/CD processes for data projects.
- Ability to create runbooks, logging, monitoring, and documentation for production exercise.
Benefits
- An innovative reality that is growing rapidly, to be built together.
- Cutting-edge projects in the world of Artificial Intelligence.
- Professional growth: collaboration with infrastructure, automation, networking, and design experts.
- Dynamic and flexible environment, with a complete and integrated Flexible Benefits system.
- Variable remuneration linked to results, Welfare, Flexible Benefits, and meal vouchers complete the package.
To apply for this job please visit linkedin.com.


Follow us on social media