Summary from listing
The Data Engineer will design, build, and maintain scalable data infrastructure and pipelines using Apache Spark, Python, and Scala. Responsibilities include data ingestion and extraction, streaming and batch processing, performance tuning, deployment on NetApp S3 and Kubernetes, containerization with Docker, unit testing, and data governance. The role requires experience with Spark, ETL, data modelling, Kafka, Docker, Kubernetes, and testing frameworks, and is based in Melbourne on a contract basis.
