Member of Technical Staff, TPU Performance Engineering at Inferact

Department: Research & Engineering

Compensation

$200,000 – $400,000/yr

Setup: On-site
Location: San Francisco, California
Type: Full-time
Level: senior
Posted: Jun 17, 2026

Description

Inferact is hiring a TPU performance engineer to build and optimize vLLM's TPU backends, compiler integrations, runtime paths, and benchmarking infrastructure using JAX, XLA, Pallas, and related tooling. The role focuses on improving correctness, latency, and throughput for production model serving on Google TPUs, with work spanning inference systems, kernels, compilers, and hardware architecture. The position is based in San Francisco, California, with remote work considered for exceptional candidates, and offers a $200,000–$400,000 USD annual salary plus equity, health, dental, vision, and 401(k) benefits.

Required Skills

JAX
XLA
Pallas
TPU execution
Performance profiling
Benchmarking
ML kernels
Compiler technologies
Quantization

Additional Information

Minimum Education: Bachelor's degree or equivalent experience in computer science, engineering, systems, machine learning, or a similar field
Required Previous Experiences: Hands-on experience building or optimizing TPU workloads using JAX, XLA, Pallas, or related compiler and runtime tooling
Experience optimizing ML kernels or inference paths, including attention, GEMM, sampling, KV cache, fused kernels, or backend runtime paths
Strong performance profiling and benchmarking skills using measurements, compiler artifacts, correctness tests, and reproducible benchmarks
Preferred Qualifications: Experience with vLLM, SGLang, TensorRT-LLM, XLA-based serving, or other LLM inference systems
Familiarity with batching, KV cache, decoding, serving tradeoffs, and backend performance constraints in production inference systems
Experience with compiler technologies such as XLA, MLIR, LLVM, Pallas, or other kernel DSLs, including lowering, fusion, and backend code generation
Knowledge of quantization methods such as INT8, FP8, mixed precision, or TPU-specific numeric formats, including accuracy and performance tradeoffs
Contribution to vLLM, JAX/XLA, Pallas, PyTorch/XLA, compiler projects, or other open-source ML infrastructure
Building TPU benchmarking infrastructure or automated performance regression detection for accelerator workloads
Work with Google TPU ecosystem stakeholders, accelerator platform teams, or early-access programs to ship backend, compiler, or inference performance improvements

Benefits & perks

Visa sponsorship

About Inferact

Inferact

Information

Inferact is a startup founded by creators and core maintainers of vLLM, an open-source LLM inference engine. Its mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster.

Open roles: 42
New in 30 days: 10
Founded: 2026

View full company profile

Get More from Metaintro

Unlock powerful job search, personalized recommendations, and deep career insights from comprehensive, market-leading data and live market signals.

For job seekers

Ready to find a role that actually fits?

Upload your résumé, start a Job Search Thread, and let Metaintro rank real openings against your experience — then guide you from search to offer.

Get Started Free Search matching jobs

Match

Compare live roles against your current evidence.

Position

Turn proof projects into role-specific applications.

Improve

Use market feedback to keep the skill plan current.

Description

Required Skills

Additional Information

Benefits & perks

About Inferact

Trending job searches

Get More from Metaintro

Ready to find a role that actually fits?

Required Skills

Additional Information

Benefits & perks

About Inferact