Summary from listing
Rootly is hiring an AI Quality Engineer to evaluate and optimize its agentic AI features. The role involves designing prompt-based test scenarios, assessing AI outputs, building evaluation frameworks, identifying hallucinations and reasoning failures, partnering with Product and Engineering on quality gates, and tracking performance metrics. Candidates should have at least five years of experience in QA, product operations, AI/ML evaluation, or a closely related role, along with hands-on experience testing LLM-powered or agentic AI products, strong prompt engineering instincts, and analytical communication skills. Benefits include medical, dental, and vision coverage, equity, vacation, a home-office stipend, and access to AI tools.
