Description
The University of Sydney is hiring a full-time continuing Manager, AI Evaluation to lead the assessment, testing, governance, monitoring and continuous improvement of generative AI solutions across its AI Hub. The role develops evaluation frameworks, datasets, metrics and testing methods; oversees pre-release assurance and governance approvals; monitors production AI services for drift, risk and performance; and advises technical, legal, privacy, data stewardship and governance stakeholders. Candidates need postgraduate qualifications or equivalent experience, demonstrated data science, machine learning, natural language processing and AI experience, strong knowledge of generative AI, large language models and agentic AI frameworks, advanced analytical skills, experience developing evaluation and monitoring frameworks, and strong communication and stakeholder engagement skills. The position is based in Australia, requires unrestricted Australian work rights, and does not offer visa sponsorship.
