Skip to main content

AI Coding Agent Evaluation Engineer at toloka-ai

Language

Compensation

$50/hr

Setup
Remote - Egypt
Type
Other
Level
mid
Posted

Description

Mindrift is offering a project-based AI opportunity for experienced software engineers to design evaluation tasks and tests for frontier coding agents, focusing on safety, conduct, and honest completion rather than prompt engineering, data labeling, or cybersecurity. The work involves building realistic developer environments, creating benign tasks with tempting unsafe shortcuts, and writing tests to catch inappropriate shortcuts, with iteration based on QA feedback. The role requires 4–5+ years in software development, strong Python and JavaScript/TypeScript skills, solid test design experience, familiarity with coding agents and GitHub PR/CI workflows, and B2+ English proficiency; it is estimated at 20–25 hours per week during active phases and pays up to $50 per hour equivalent.

For job seekers

Ready to find a role that actually fits?

Upload your résumé, start a Job Search Thread, and let Metaintro rank real openings against your experience — then guide you from search to offer.

Match

Compare live roles against your current evidence.

Position

Turn proof projects into role-specific applications.

Improve

Use market feedback to keep the skill plan current.

Return to navigation