Data Engineer

WSP India is seeking a Data Engineer to support the AU business through GCC. The role involves designing, building, and maintaining the data engineering layer of the Microsoft Fabric platform. The successful candidate will shape the platform from the ground up, making foundational decisions on standards and patterns that will scale across the organisation.

Requirements

• Build and maintain Data Factory pipelines for API-based data ingestion
• Develop Python ETL scripts for custom extraction, API polling, and file ingestion
• Develop and maintain Spark/Python notebooks and SQL-based transformation logic
• Create Dataflows Gen2 for handling Excel/CSV-based messy data
• Implement incremental loads, watermarking, and CDC patterns
• Integrate data from Dataverse and other enterprise sources into Fabric
• Own Bronze → Silver transformations (cleansing, deduplication, standardisation)
• Support Silver → Gold transformations (star schema, fact/dimension modelling)
• Prepare feature datasets for AI/ML workloads and support notebook-based model development
• Implement data quality frameworks (validation rules, anomaly detection)
• Build and optimise Power BI semantic models
• Prepare datasets for reporting and basic analytics use cases
• Monitor pipeline performance, platform health, and Fabric capacity
• Maintain documentation and manage Fabric capacity

Benefits

• Flexible work environment
• Opportunities for career growth and development
• Global recognition for health and safety practices
• Inclusive and diverse work environment