Senior Inference Engineer at qodeworld

Language

Location: Vietnam
Level: not_specified
Posted: Jul 23, 2026

Description

The Senior Inference Engineer will focus on building and optimizing inference engines for large-scale LLM serving systems. This involves working across various layers of the inference stack, analyzing bottlenecks, implementing optimization techniques, and collaborating with platform engineering teams to enhance serving architectures.

Required Skills

Software Engineering
Distributed Systems
Performance Optimization
Python
Go
C++

Additional Information

Required Previous Experiences: At least 5 years of experience as a Software Engineer, Performance Engineer, or equivalent.
Experience developing or optimizing distributed systems, high-throughput backends, or large-scale serving systems.
Experience with benchmarking, profiling, and performance tuning in production environments.
Preferred Qualifications: Experience with Linux internals, kernel tuning, or custom Linux kernel.
Understanding of GPU Architecture or CUDA Programming.
Experience with AI/ML Serving Systems or LLM Inference.
Have worked with one of the inference engines such as vLLM, SGLang, TensorRT-LLM, or Triton Inference Server.
Understanding of batching, KV Cache, quantization, speculative decoding, tensor/pipeline parallelism, or disaggregated serving.
Experience with the NVIDIA inference stack (TensorRT, Triton, CUTLASS, NCCL, cuBLAS, cuDNN).
Experience with observability stacks such as Prometheus, Grafana, or OpenTelemetry.
Open-source contributions or research related to AI Infrastructure, ML Systems, or Performance Optimization.