Description
Google is hiring a Software Engineer to design, develop, and optimize software components for Large Language Model inference serving on the Google Distributed Cloud platform. The role covers model lifecycle management, data loading, request routing, load balancing, disaggregated serving, speculative decoding, quantization, model sharding, and integration with billing, logging, observability, security, quota management, Kubernetes, networking, and hardware acceleration. Candidates need a bachelor’s degree or equivalent practical experience, at least five years of software development experience, and experience with Go and large-scale distributed systems.
