Description
The Data Engineer will design, build, and optimize scalable ETL/ELT pipelines using Python and PySpark on Azure Databricks and Azure Data Factory. Responsibilities include data ingestion, transformation, quality monitoring, SQL and data modeling, infrastructure management on OpenShift, CI/CD automation with GitHub Actions, performance optimization, and collaboration with data science and engineering teams. The role requires 6–8 years of relevant experience and strong proficiency in Python, PySpark, SQL, Azure data services, containerization, and observability tools.
