Skip to main content

AI Coding Agent Evaluator at toloka-ai

Language

Compensation

$0 – $40/hr

Setup
Remote - Costa Rica
Type
Contract
Level
mid
Posted

Description

Mindrift is seeking experienced software developers for a project-based AI coding-agent evaluation opportunity. The role involves building realistic simulated developer environments, designing challenging tasks and success criteria, writing functional and integration tests that fairly evaluate AI-generated solutions, and iterating based on QA feedback. Candidates should have 5+ years of software development experience, proficiency with Python/FastAPI, JavaScript or TypeScript/React, Docker, Postgres, Kafka, Redis, and English at B2+ level. Participation is project-based rather than permanent employment, with tasks estimated at approximately 20 hours and compensation up to $40 per hour equivalent.

Trending job searches

Every query opens live roles, salary samples, and market demand — tap a search to run it instantly.

Get More from Metaintro

Unlock powerful job search, personalized recommendations, and deep career insights from comprehensive, market-leading data and live market signals.

For job seekers

Ready to find a role that actually fits?

Upload your résumé, start a Job Search Thread, and let Metaintro rank real openings against your experience — then guide you from search to offer.

Match

Compare live roles against your current evidence.

Position

Turn proof projects into role-specific applications.

Improve

Use market feedback to keep the skill plan current.

Return to navigation