Evaluations Engineering - Member of Technical Staff at simile

Department: Engineering

Compensation

$200,000 – $400,000/yr

Setup: On-site
Location: San Francisco, California
Type: Full-time
Level: senior
Posted: Jul 17, 2026

Description

Simile is hiring a Member of Technical Staff in Evaluations Engineering to build systems that evaluate the accuracy, trustworthiness, and improvement of human-behavior simulations. The role covers evaluation execution infrastructure, data systems, versioning, provenance, access controls, customer validations, survey operations, human-data workflows, labeling and review tools, and interfaces for model comparison and regression detection. Candidates should have several years of experience building production software, backend services, data pipelines, and relational data models, along with familiarity with modern model-development and evaluation workflows. The position offers a $200,000–$400,000 USD salary range, equity, and medical, dental, and vision coverage.

Required Skills

Backend services
Data pipelines
Relational data models
Evaluation infrastructure
Workflow orchestration
LLM evaluation
Human-data systems
Access controls

Additional Information

Required Previous Experiences: Several years of experience building and maintaining production-quality software
Experience building backend services, data pipelines, automation workflows, and relational data models
Preferred Qualifications: Experience building LLM or ML evaluation systems, benchmark platforms, regression suites, experiment-tracking tools, or model-quality dashboards
Experience developing technical surfaces for ML engineers, researchers, data scientists, or operations teams
Experience with labeling platforms, expert-review workflows, LLM-as-judge systems, grader calibration, or other human-in-the-loop evaluation methods
Experience automating surveys, experiments, customer-data ingestion, or other human data collection workflows
Comfort reasoning about sampling error, uncertainty, calibration, confidence intervals, and distributional metrics
Experience designing permissions, auditability, and data-governance systems for human or customer data
Experience using modern AI coding tools to accelerate development while independently testing and validating their output