Staff AI Runtime Engineer at FlexAI

Location: San Jose, California
Level: Principal
Posted: Mar 24, 2026

Description

FlexAI is seeking a Staff AI Runtime Engineer to design, develop, and optimize the core runtime infrastructure for distributed training and deployment of large AI models. This hands-on leadership role involves owning critical components of the PyTorch-based stack, leading technical direction, and collaborating across various teams to drive innovation in elastic, fault-tolerant, high-performance model execution.

Required Skills

Systems Software Engineering
AI Runtime
Distributed Systems
Compiler/Runtime Interaction
PaaS Services
Deep Learning Runtimes
PyTorch
TensorFlow
JAX
Python
C++
Go
Rust
Distributed Training Frameworks
Low-Level Performance Tuning
Resource Orchestration
Multi-GPU
Multi-Node
Cloud-Native AI Workloads
Containerized Workloads
Job Scheduling
Failure Recovery

Additional Information

Required Previous Experiences: 8+ years of experience in systems/software engineering, with deep exposure to AI runtime, distributed systems, or compiler/runtime interaction.
Experience in delivering PaaS services.
Proven experience optimizing and scaling deep learning runtimes (e.g. PyTorch, TensorFlow, JAX) for large-scale training and/or inference.
Preferred Qualifications: Contributions to PyTorch internals or open-source DL infrastructure projects.
Familiarity with LLM training pipelines, checkpointing, or elastic training orchestration.
Experience with Kubernetes, Ray, TorchElastic, or custom AI job orchestrators.
Background in systems research, compilers, or runtime architecture for HPC or ML.
Start up previous experience

About FlexAI

FlexAI

AI infrastructure1-10

FlexAI is a vertical NeoCloud (AI-native managed infrastructure) offering managed inference on 20+ open models, an Agent SDK in trial, and AI Factory private cloud deployments, at competitive per-token pricing you can verify.

Customers and case studies

See customers and case studies