AI Coding Agent Evaluation Specialist at toloka-ai

Mindrift is recruiting experienced software developers for a project-based opportunity to create and evaluate challenging tasks for AI coding agents. The work involves building realistic simulated developer environments, designing solvable tasks and acceptance criteria, writing functional and integration tests that distinguish valid from invalid agent solutions, and iterating based on QA feedback. Applicants should have at least eight years of software development experience, proficiency with Python/FastAPI, JavaScript or TypeScript/React, Docker, PostgreSQL, Kafka, and Redis, plus B2+ English proficiency. Tasks are estimated at about 30 hours each, completed flexibly by deadline, and paid per accepted task at a rate of up to the equivalent of $200 per hour.