Design, build, and maintain end-to-end data pipelines for ingestion, transformation, and delivery of large-scale data. Develop and optimize data processing logic using PySpark on Databricks. Implement ETL/ELT pipelines integrating data from multiple structured and semi-structured sources.
Requirements
- Design and build end-to-end data pipelines
- Develop and optimize data processing logic using PySpark on Databricks
- Implement ETL/ELT pipelines
- Ensure data quality, reliability, performance, and observability
- Optimize Spark jobs through partitioning, caching, and performance tuning
- Collaborate with data architects, analysts, and business stakeholders
- Implement best practices in CI/CD, version control, and pipeline automation
- Support the evolution of modern data platforms and analytics capabilities
- Work with orchestration tools
- Communicate technical solutions and trade-offs effectively
Benefits
- Health Insurance
- Retirement Plan
- Paid Time Off
- Stock Options
- Life Insurance
- Dental Insurance
- Vision Insurance
To apply for this job please visit fa-etqd-saasfaprod1.fa.ocs.oraclecloud.com.
Follow us on social media