Genpact is inviting applications for the role of Principal Consultant, Sr. Databricks Developer. In this role, the Sr. Databricks developer is responsible for solving real world cutting edge problems and mentoring a team of junior developers. Responsibilities include designing and developing end-to-end data pipelines, translating requirements into scalable data solutions, participating in architecture discussions, and implementing data quality checks.
Requirements
- Design and develop end-to-end data pipelines using PySpark, SQL/Spark-SQL, and Delta Lake.
- Translate requirements into scalable, high-performance data solutions.
- Participate in architecture discussions and provide technical inputs.
- Implement ETL/ELT workflows in Lakehouse (Bronze, Silver, Gold layers).
- Optimize pipelines, clusters, and queries for performance and cost.
- Integrate Databricks with cloud services, APIs, and messaging systems.
- Implement data quality checks, unit tests, and integration tests.
- Develop both batch and streaming pipelines for real-time use cases.
- Contribute to shared assets, frameworks, and accelerators.
- Collaborate with leads, architects, and analysts during delivery.
- Support data migration projects to Databricks Lakehouse.
- Document technical solutions, coding standards, and best practices.
- Act as a strategic advisor on Databricks adoption, modernization, and data strategy.
- Define enterprise-scale architectures leveraging the Databricks Lakehouse platform.
- Establish data governance frameworks with Unity Catalog (lineage, auditing, security, compliance).
- Oversee multiple delivery streams, ensuring alignment with standards and best practices.
- Drive reusable accelerators, frameworks, and reference architectures.
- Engage with stakeholders on cost optimization, innovation, and roadmap planning.
- Lead thought leadership initiatives (whitepapers, blogs, conferences).
- Mentor and coach developers, senior developers, and leads.
- Support Centers of Excellence (CoE) and competency frameworks.
- Stay ahead of emerging technologies, trends, and Databricks innovations.
Benefits
- Generous Paid Time Off
- 401k Matching
- Retirement Plan
- Tuition Reimbursement
- Relocation Assistance
Follow us on social media