AI Coding Agent Evaluation Specialist at toloka-ai

Mindrift is recruiting experienced software developers for a project-based opportunity to create and evaluate benchmark tasks for AI coding agents. The work involves building realistic simulated development environments, designing intermediate-state coding tasks and acceptance criteria, writing functional and integration tests, and iterating based on QA feedback. It is not permanent employment, data labeling, prompt engineering, or primarily writing code from scratch. The opportunity requires 8+ years of software development experience, proficiency with Python/FastAPI, JavaScript or TypeScript/React, Docker, Postgres, Kafka, Redis, testing, and English at B2+ level. Work is paid per accepted task, with tasks estimated at 30 hours and compensation up to the equivalent of $200 per hour.