Machine Learning Engineer at Together AI

Compensation

$160,000 – $230,000/yr

Location: San Francisco, California
Type: Full-time
Posted: May 7, 2026

Description

Together AI is looking for a Machine Learning Engineer to join their Inference Engine team. The role focuses on optimizing and enhancing the performance of AI inference systems, particularly using state-of-the-art large language models like those powered by PyTorch. The engineer will design and build production systems, develop optimized runtime inference services, and collaborate with various teams to deliver cutting-edge AI solutions.

Required Skills

Python
PyTorch
multi-threading
memory management
networking
storage

Additional Information

Required Previous Experiences: 3+ years of experience writing high-performance, well-tested, production-quality code
Preferred Qualifications: Knowledge of existing AI inference systems such as TGI, vLLM, TensorRT-LLM, Optimum
Knowledge of AI inference techniques such as speculative decoding.
Knowledge of CUDA/Triton programming.
Knowledge of Rust, Cython and compilers.