Description
This role involves designing and developing robust ingestion, transformation, and enrichment pipelines using Python, PySpark, and SQL. The individual will write and optimize complex SQL queries, collaborate with data architects and scientists, unit-test and integrate-test code, maintain CI/CD pipelines, monitor production workloads, and document data lineage. They will also contribute to innovation, participate in Agile ceremonies, mentor junior engineers, and evaluate emerging technologies.
