Principal Architect, AI Inference Networking - NIXL and Dynamo at NVIDIA

Department: Architect

Language

Setup: On-site
Location: Huangpu, Shanghai · Shenzhen, Guangdong Province
Type: Full-time
Level: senior
Posted: Sep 23, 2026

Description

NVIDIA is hiring a Senior Inference Architect to drive NIXL adoption in China, profile customer inference stacks, fix throughput and time-to-first-token issues, write NIXL backends and plugins, extend Dynamo integrations, define dynamic data-path APIs, and contribute optimizations upstream. The role requires an M.Sc. or Ph.D. in a relevant field, 12+ years of experience building or optimizing large-scale distributed systems, strong C++ and Python systems programming, RDMA and GPU memory expertise, modern LLM serving knowledge, and fluent Mandarin and English. The architect will work with core NIXL and Dynamo engineers in Israel and the United States.

Required Skills

NIXL
Dynamo
C++
Python
RDMA
GPU memory
LLM serving
Distributed systems

Additional Information

Minimum Education: M.Sc. or Ph.D. in Computer Science, Electrical Engineering, or Computer Engineering, or equivalent depth earned in industry
Required Previous Experiences: 12+ years building or optimizing large-scale distributed systems, including AI inference or training infrastructure, communication libraries, high-performance networking, or HPC runtimes
Hands-on experience with RDMA networking and GPU memory movement
Working knowledge of modern LLM serving, disaggregated inference, KV cache management, tensor and pipeline parallelism, and at least one serving stack
Preferred Qualifications: Open-source contributions to NIXL, Dynamo, NCCL, vLLM, SGLang, TensorRT-LLM, or Mooncake
Experience with inference at scale, including thousands of GPUs, multi-tenant clusters, and strict SLOs
CUDA experience and comfort reading kernel and driver-level code
Experience building an adopted project or a cited talk
Translation Company Name: NVIDIA

About NVIDIA

NVIDIA

人工智能计算Santa Clara, CA, US

NVIDIA invents the GPU and drives advances in AI, high-performance computing, gaming, creative design, autonomous vehicles, and robotics. Its accelerated computing, libraries, and AI models enable software makers and industry manufacturers to develop products using large-scale, physically accurate simulations and interactive digital twins.

Customers and case studies

See customers and case studies