Staff Software Engineer, Inference API at Cerebras Systems

Department: Software Engineering

Language

Setup: Hybrid
Location: Toronto, Ontario
Type: Full-time
Level: senior
Posted: Sep 23, 2026

Description

Cerebras Systems is hiring a Software Engineer to build and evolve the machine-learning API layer for its heterogeneous AI inference platform. The role focuses on production ML inference APIs, model integration, request routing, API compatibility, distributed serving, performance, reliability, observability, testing, and developer experience across GPU prefill and Cerebras decode backends. Candidates need substantial software engineering experience, strong Python and Go skills, experience with distributed systems and inference APIs, and a bachelor's degree or equivalent practical experience.

Required Skills

Python
Go
Distributed systems
API design
Model serving
Request routing
Observability
Kubernetes
vLLM

Additional Information

Minimum Education: Bachelor's degree in computer science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.
Required Previous Experiences: 5+ years of software engineering experience, including substantial individual-contributor ownership of production software or distributed systems.
Experience building stable APIs with validation, error handling, observability, compatibility, and versioning practices.
Experience integrating software across service, framework, runtime, and infrastructure boundaries.
Experience designing or maintaining OpenAI-compatible, gRPC, REST, or streaming inference APIs.
Experience with Linux, containers, Kubernetes or comparable orchestration systems, CI/CD, and operating latency-sensitive services in production.
Ability to diagnose correctness, reliability, and performance issues across distributed serving systems.
Preferred Qualifications: Experience modifying or contributing to open-source machine learning systems projects.
Experience creating API conformance, model-quality, numerical-comparison, determinism, or performance-regression test systems.
Experience building SDKs, developer tools, model registries, configuration systems, or self-service ML platforms.
Experience with multi-model or multi-tenant inference platforms.
Understanding of model-specific tokenization, generation configuration, logits processing, tool calling, structured generation, and constrained decoding.
Experience with disaggregated prefill/decode architectures and request scheduling.
Experience designing, building, or operating production APIs and services for machine learning or data-intensive applications.
Familiarity with reduced-precision inference and quantization formats.