Skip to main content

Software Engineering Evaluation Specialist at toloka-ai

Compensation

$35/hr

Setup
Remote - Lithuania
Type
Contract
Level
not_specified
Posted

Description

Mindrift is seeking a Software Engineering Evaluation Specialist to design challenging coding tasks for AI agents. The specialist will create self-contained Docker environments with broken software pieces, allowing AI agents to attempt fixes verified by automated tests. Responsibilities include inventing realistic developer scenarios, building reproducible Docker environments, writing robust pytest scripts, creating clear instructional documents, and providing reference solutions. This role focuses on designing problems for AI agents rather than producing production code.

For job seekers

Ready to find a role that actually fits?

Upload your résumé, start a Job Search Thread, and let Metaintro rank real openings against your experience — then guide you from search to offer.

Match

Compare live roles against your current evidence.

Position

Turn proof projects into role-specific applications.

Improve

Use market feedback to keep the skill plan current.

Return to navigation