Description
The Data Engineer role focuses on manipulating data in a software engineering capacity, including normalizing databases, constructing consistent machine-readable datasets, and combining information from multiple sources. The role requires deep SQL, data modeling, and experience with Spark, Hive, Airflow, Databricks pipelines, batch and real-time processing, data quality tools, data engineering and analytics solution design, big data technologies, statistical and AI/ML solutions, containerization, Agile, DevOps, CI/CD, and API-based data sourcing, enrichment, and delivery. It also specifies hands-on experience with on-premises and Microsoft cloud-based data engineering and analytics tools.
