Anyscale

“With Anyscale, we've achieved 20% better latency compared to our previous solution – all while supporting continued user growth.”

“Anyscale made horizontal scaling easy out of the box. It gave us seamless scalability to keep up with our fast-growing, low-latency inference needs – all without adding operational burden.”

“At Notion, we knew our rapid growth would push our infrastructure beyond its limits. We needed a solution that could scale horizontally with our growth while maintaining strict low-latency performance requirements for our users. Anyscale was the answer.”

“Anyscale gives us the confidence that when we build, it just works. It removes the friction around environment management and scaling, so our teams can focus on delivering fast, intelligent experiences to our users.”

“With Anyscale, we can tap into on-demand A100 and H100 capacity across clouds to support our expanding R&D efforts and the growing volume of training runs they require.”

“GPU availability in the cloud is quite scarce and a primary bottleneck for us, as we needed access to premium GPUs like H100s on-demand. The ability to implement a multi-cloud solution has been instrumental in accelerating our AI development.”

“Our customers are extremely sensitive about their data. Anyscale lets us run distributed training in our own cloud environment, so we can keep our customers’ data in our own hands.”

“By offloading Ray cluster operations to Anyscale, our engineers can concentrate on pushing the boundaries of physical AI rather than managing distributed compute infrastructure.”

“Using Anyscale as our multi-cloud Ray cluster manager, we’ve made our training runs fully portable across hyperscalers and specialized cloud providers, maximizing GPU availability.”

“We had these new robotic behaviors we needed to onboard onto our product to meet rising customer demand, but the AI infrastructure that got us here couldn’t scale to handle the more complex models we needed for the next phase.”

“Before, sellers would take 28 days to fully onboard. With Catalog Transformer and Anyscale, we reduced that to just a few hours in most cases.”

“With GitHub Actions and the Anyscale CLI, we could deploy Ray clusters as code and manage three isolated environments for dev, test, and production. It gave our data scientists the autonomy they needed.”

“Using GPT-4 would have cost us more than half a million dollars per month… with Anyscale and fine-tuned LLaMA models, we brought that down by 3× while scaling to millions of products.”

“We wanted a solution that was easy to run and operate, while having more control over the LLMs we could run – and Anyscale gave us that.”

“What we’re doing would be prohibitively expensive with other AI platforms. Anyscale  enables us to quickly take advantage of open models like Llama-3, Mixtral, or whatever comes next, and in AI, there’s always a ‘next’. We’re ready for it with Anyscale.”

“Support was excellent during the trial period and continues to be excellent. We deployed and fine-tuned Endpoints on Llama. We call any time something breaks.  We get tips about specific optimizations regarding batch-size, compute configurations, tensor transformations. Support is always there and they respond quickly.”

“Anyscale enables our small ML infra team to support state-of-the-art ML solutions at scale on accelerated hardware. Whether it’s deploying the latest generative AI models or more general ML solutions, Anyscale relieves the burden on ML infra—allowing our team to focus on other impactful work.”

“Iterating and scaling foundational embedding models (graph neural networks, two tower recommenders) trained over 100M-200M+ interactions was difficult and time-consuming until we migrated our feature producers and training jobs to Anyscale + Ray.  Our experiment velocity with deep models and dependencies has 5x’ed while training on more data for cheaper. Anyscale has also enabled net-new real-time inference services and real-time user experiences that data scientists can roll out without waiting on ML Ops experts.”

“Using Anyscale as a push-button solution for managing Ray clusters enabled data scientists to quickly set-up and scale production environments with deep learning dependencies—even for real-time applications that typically would require significant ML Ops experience.”

“With Anyscale, we have no ceiling on scale, and an incredible opportunity to bring AI features and value to our 170 million users”

“Our new platform gives us optionality everywhere. Nothing’s off the table and our nimble approach means we can evolve as AI does. We have no ceiling on scale, and an incredible opportunity to bring AI features and value to our 170 million users.”

“We've been able to innovate quickly and successfully, and user feedback is very positive.”

“" One of our applied AI engineers said, ‘we should use this model,’ and the next day it was running in production. Before Anyscale, that would’ve taken a week or more. "”

“" There’s like a quarter of the code or less in our Anyscale service definitions versus our self-hosted deployment. I love deleting code. The less code the better. "”

“" Engineers can look at these metrics to quickly see what’s going on. It helps our MLOps team debug faster. "”