Staff Engineer, AI Testing and Evaluation at idme

Compensation

$217,565 – $271,000/yr

Location: Mountain View, California
Type: Full-time
Level: staff+
Posted: Sep 2, 2026

Description

ID.me is hiring a Staff Engineer to define and lead testing and evaluation practices for AI agents and LLM-powered systems. The role owns AI quality standards, evaluation infrastructure, production observability, test strategies, developer tooling, and cross-team quality gates, while mentoring engineers and partnering with Security, Platform, Product, and AI/ML teams. Candidates need a bachelor's degree or equivalent experience, at least 8 years of production software experience, LLM and agent testing experience, strong backend engineering fundamentals, and experience with eval frameworks and agentic frameworks. The position is based in Mountain View, California, with an annual base salary of $217,565 to $271,000 USD.

Required Skills

RAG pipelines
Evaluation infrastructure
Python
LangChain
Red-teaming
Observability

Additional Information

Minimum Education: Bachelor's degree in Computer Science, Engineering, or equivalent experience
Required Previous Experiences: 8+ years building and operating production software systems
Experience evaluating or testing LLM-powered features or autonomous agents in production
Experience designing test infrastructure, CI/CD quality gates, or evaluation pipelines at scale
Experience improving developer experience through internal tooling and workflow acceleration
Experience building eval frameworks for LLM agents
Red-teaming or adversarial testing experience for AI models or agents
Preferred Qualifications: Background in identity verification, fraud detection, or regulated industries
Familiarity with Anthropic's model evaluation methodology or similar published eval research
Experience with observability tooling applied to AI workloads
Track record of building developer tooling or platforms adopted widely