Anyscale Partners with NVIDIA to Scale Generative AI Models into Production

SAN FRANCISCO, March 19, 2024 — Anyscale, the AI infrastructure company built by the creators of Ray, the world’s fastest growing open-source unified framework for scalable computing, announced a collaboration with NVIDIA to integrate the NVIDIA AI Enterprise software platform into the Anyscale platform, enabling customers to accelerate and scale large language models (LLMs) into a production environment with security, support, and stability.

The integration brings support for NVIDIA NIM inference microservices announced at NVIDIA GTC this week. Customers will benefit from the combined power of Ray and Anyscale’s managed runtime environment, providing capabilities like container orchestration, observability, and autoscaling, as well as access to NVIDIA AI Enterprise to improve security and LLM performance.

Increasingly, AI workloads demand more performance from infrastructure. Dynamically scaling that infrastructure while balancing cost remains a pervasive challenge. Anyscale’s integration with NVIDIA AI Enterprise will enhance the scalability of AI workloads, enabling training and deployment of larger and more complex models and support the optimization of smaller models for specific tasks. Access to NVIDIA’s accelerated computing infrastructure will simplify the deployment and management of distributed machine learning (ML) applications, leading to more efficient resource utilization, faster iteration, and reduced costs.

In addition, access to NIM minimizes restrictions and time spent on infrastructure, allowing developers to remain focused on driving innovation forward for their organization.

“This enhanced integration with NVIDIA AI Enterprise makes it simpler than ever for customers to get access to cutting-edge infrastructure software and top-of-the-line compute resources to accelerate the production of generative AI models,” said Robert Nishihara, CEO of Anyscale. “As AI becomes a strategic capability, it’s essential to balance performance, scale, and cost while minimizing infrastructure complexity. The ability to tap into the best-of-breed infrastructure, accelerated computing, pre-trained models and tools will be crucial for organizations to stand out and compete. This collaboration is another important step forward in bringing generative AI to more people.”

“Enterprises need full stack computing to deliver low latency, high performance generative AI,” said Justin Boitano, vice president, Enterprise Products at NVIDIA. “The integration of NVIDIA NIM microservices with Anyscale’s platform simplifies and accelerates generative AI development and deployment, bringing more efficiency to production AI.”

Anyscale’s integration with NVIDIA AI Enterprise will deliver multiple benefits for developers, including:

• Auto-scaling to reduce costs: Through auto-scaling capabilities, users can achieve improved resource utilization to reduce costs while simultaneously enhancing the performance and scalability of their AI applications.
• Observability: Full transparency delivers insights into application performance and reliability, enabling developers to adjust their code in order to optimize system performance.
• Optimization: Customers have access to generative AI-optimized inference via NVIDIA NIM, allowing the development of better solutions for specific use cases such as audio processing or medical imaging.
• Security: Developers have the ability to tap into advanced security features and support from Anyscale and NVIDIA, enabling the use of proprietary data to train models safely and securely.
• Workspaces: Collaborative development through workspaces will provide customers with faster iteration and deployment of models and the ability to streamline workflows.

Anyscale is now available to NVIDIA AI Enterprise users at www.anyscale.com/get-started.

About Anyscale

Anyscale is the leading AI application platform. With Anyscale, developers can build, run and scale AI applications instantly. Built by the creators of Ray, the world’s fastest growing open-source unified framework for scalable computing, thousands of companies rely on technology from Anyscale to accelerate the delivery of AI products to market at significantly reduced cost. Backed by Andreessen Horowitz, NEA, Addition, Intel Capital and Foundation Capital, Anyscale is headquartered in San Francisco, CA.

Source: Anyscale