AI Coding Agent Evaluation Specialist at toloka-ai

Mindrift is recruiting experienced software developers for a project-based opportunity to create and evaluate tasks used to test AI coding agents. The work involves building realistic simulated developer environments, designing solvable coding tasks and evaluation criteria, writing functional and integration tests, and refining tasks based on QA feedback. Candidates need 8+ years of software development experience, proficiency with Python/FastAPI, JavaScript or TypeScript/React, Docker, Postgres, Kafka, Redis, testing, and English at B2+ level. Tasks are estimated at about 30 hours each, are completed flexibly by deadline, and are paid per accepted task at rates up to the equivalent of $200 per hour.