Rockset Primes Database for Massive Vector Serving
Rockset today unveiled new vector database capabilities, such as the addition of approximate nearest neighbor (ANN) search and native support for LlamaIndx and LangChain, that it says will help companies efficiently scale their GenAI applications once they’re in production.
As companies experiment with the new generative AI capabilities delivered via large language models (LLMs) and vector search, they’re getting good early results, says Rockset co-founder and CEO Venkat Venkataramani.
“We’re not educating people on what can vector search do for you,” he says. “They’ve already tinkered it at very small scale, built prototypes, and they already see the magic.”
While vector search and GenAI prototypes tease a tantalizing future, companies often run into trouble when they try to make the leap from development to production.
“Not a week goes by where somebody calls me and says, ‘Venkat, I started with this toy open source vector database and we did a shadow launch and a scale test, and it just bombed,’” Venkataramani says. “Other vector databases may have good vector support, but the database part is very shaky. Is it scalable? Is it reliable? It gets very expensive and very hard to operate very quickly.”
Rockset rolled out its initial support for vector search and storing vectorized embeddings earlier this year. Like many other SQL and NoSQL databases, the Silicon Valley firm experienced a surge in demand for these data types, which are instrumental for enabling vector search as well as other types of GenAI applications built atop LLMs and computer vision models.
The addition today of ANN and native support for LlamaIndex and LangChain, which are open source tools for automating prompt engineering and other critical behind-the-scenes GenAI data workflows, bolster Rocket’s existing capabilities for serving scalable GenAI apps.
The ANN algorithm is critical for quickly matching GenAI app user input to pre-generated vector embeddings stored in a vector database. It’s used both in vector search, where it powers the similarity search, as well as other GenAI use cases for text and computer vision.
Rocket’s implementation of ANN is unique, Venkataramani says, because it rebuilds the ANN index in real time as new data arrives, versus as a batch job that requires downtime.
“Other vector databases require you to rebuild the entire ANN index and all of that in batch mode, and so you don’t really get a real time application,” he says. “Rebuilding these indexes also is actually way more computationally expensive, but if you can incrementally maintain it, it is a lot cheaper and also more real-time.”
Rockset’s support for compute-compute separation enables it to run workloads such as index rebuilding, compaction, and ongoing maintenance without impacting the application’s main vector query workload, Venkataramani says. Compute-compute separation gives the database a big advantage when it comes to scaling GenAI applications, he says.
“You can have one or more compute instances for searches and similarity searches and vector searches and other real-time analytics and reporting–whatever applications you have,” the Datanami 2022 Person to Watch says. “They’re completely decoupled. They’re fully independently scalable and isolated from each other. But they work on the same copy of the data, and new data coming in–new updates, inserts, and deletes–will be available for your searches within single-digit milliseconds.”
The fact that Rockset, as a distributed relational database, can store all of a customer’s data as opposed to just storing vectors, as a dedicated vector database does, is another big advantage, Venkataramani says.
“You can have one column that’s basically vector embeddings, and all the other columns and other structured data available right there,” he says. “Building these kinds of hybrid searches across vectors and other metadata that you have is as simple as a SQL where clause. It’s not like you have a vector database and then you put all the other metadata and other structured data in a second separate database and you have to somehow in the application wire them together.”
Having all of the data in one place turns out to be very important in some GenAI use cases, such as powering a song recommendation engine, Venkataramani says. Running the ANN or K nearest neighbor (KNN) search–which applies a brute-force approach that delivers exact answers–is just one step among many that happens behind the scenes in recommendation engine. Developers may also bring some pre- and post-filtering using other metadata to get the best song recommendations in front of the user.
“You want to push the computation close to where the data lives, but the optimizer needs to be able to know which filters to apply first and which filters to apply second,” he says. “Imagine I have all the vectors in the vector database and all the metadata in the second database. Which one do I do first? If I go and get the 10 songs that are closest in the vector database, all of them might be in my recent playlist. If I go and look at all the songs from all these artists, none of them might be nearest neighbors. So I have to be able to combine them in the same SQL WHERE clause to be able to do this efficiently on the same data set.”
Since OpenAI ignited the GenAI storm a year ago with the launch of ChatGPT, the need for vector capabilities has exploded in the database market. Rockset’s vector capabilities are attracting attention among existing customers as well as prospects that are building GenAI applications, ranging from chatbots to recommendation engines to vector search, Venkataramani says.
“It’s really hot. It’s very, very significant,” he says. “AI applications are not like…a separate category of apps. Every application will have parts of their application powered by AI models and AI kind of capabilities, and it’ll be invisible…You’re not going to have a separate one-off side database to build your AI apps. Every single app in the world right now is going to get enhanced and have some components of it.”
One of the companies adopting Rockset’s vector capabilities is JetBlue. The airline, which recently shared its participated in the vendor’s one-day conference, did a bake-off between Rockset and several other vector database, and picked Rockset to power GenAI and other applications.
“We saw the immense power of real-time analytics and AI to transform JetBlue’s real-time decision augmentation and automation, since stitching together three to four database solutions would have slowed down application development,” Sai Ravuru, JetBlue’s senior manager of data science and analytics, says in a recent case study. “With Rockset, we found a database that could keep up with the fast pace of innovation at JetBlue.”
December 7, 2023
- Dell Technologies Boosts AI Performance with Advanced Data Storage and NVIDIA DGX SuperPOD Integration
- Intel Labs to Present New AI Research at NeurIPS 2023
- VAST Data Closes Series E Funding Round, Nearly Triples Valuation to $9.1B
- Sprinklr Empowers Businesses to Deploy and Scale Generative AI-powered Conversational Bots
- KNIME Releases Improved UI, Enhanced AI Assistant, Modernized Scripting Experience with AI, and More
- EY Report Highlights: Generational Divide in AI Adoption and Perception in the Workforce
- Bigeye Receives Strategic Investment from Alteryx Ventures
December 6, 2023
- Astronomer Unveils Latest Astro Release with Advanced Security and Cost-Savings Features
- Asato Secures $7.5M Investment to Support Development of AI Copilot Platform
- AMD Instinct MI300 Series Launch: Accelerating Next-Gen AI and Supercomputing
- SQream Achieves SOC-2 Type II Compliance Certification for Its Cloud-Native Data Lakehouse ‘Blue’
- Ataccama Announces ONE AI for Improved Automated Data Governance
- 10% of Organizations Surveyed Launched GenAI Solutions to Production in 2023
- SingleStore to Launch Hybrid Vector and Full-Text Search Capabilities as a Snowflake Native App on the Snowflake Data Cloud
- Snowplow Launches Snowplow Digital Analytics as a Snowflake Native App, in the Data Cloud
- Hitachi Vantara Launches Unified Compute Platform Integrated with GKE Enterprise to Simplify Hybrid Cloud Management
- Red Hat Reports: IT Modernization and Open Source Adoption Key to Overcoming Skills Shortfalls
December 5, 2023
- Nexusflow Unveils NexusRaven-V2, Offering Advanced Software Tool Use Beyond GPT-4 Capabilities
- Alteryx Research Outlines the Challenges Facing the Enterprise of the Future
- Unravel Data Partners with Databricks for Lakehouse Observability and FinOps
Most Read Features
- Databricks Bucks the Herd with Dolly, a Slim New LLM You Can Train Yourself
- Big Data File Formats Demystified
- Altman’s Back As Questions Swirl Around Project Q-Star
- Data Mesh Vs. Data Fabric: Understanding the Differences
- Quantum Computing and AI: A Leap Forward or a Distant Dream?
- Patterns of Progress: Andrew Ng Eyes a Revolution in Computer Vision
- AWS Adds Vector Capabilities to More Databases
- Taking GenAI from Good to Great: Retrieval-Augmented Generation and Real-Time Data
- Five AWS Predictions as re:Invent 2023 Kicks Off
- How Generative AI Is Transforming the Call Center Market
- More Features…
Most Read News In Brief
- Mathematica Helps Crack Zodiac Killer’s Code
- Databricks: We’re a Data Intelligence Platform Now
- Pandas on GPU Runs 150x Faster, Nvidia Says
- GenAI Debuts Atop Gartner’s 2023 Hype Cycle
- Retool’s State of AI Report Highlights the Rise of Vector Databases
- Amazon Launches AI Assistant, Amazon Q
- AWS Launches High-Speed Amazon S3 Express One Zone
- New Data Unveils Realities of Generative AI Adoption in the Enterprise
- Big Growth Forecasted for Big Data
- Anaconda’s Commercial Fee Is Paying Off, CEO Says
- More News In Brief…
Most Read This Just In
- Salesforce Announces New Automotive Cloud Features
- Martian Raises $9M for Advanced Model Mapping to Enhance LLM Performance and Accuracy
- DataStax Launches New Integration with LangChain, Enables Developers to Build Production-ready Generative AI Applications
- Dremio Delivers GenAI-Powered Data Discovery and Unified Path to Apache Iceberg on the Data Lakehouse
- HPE Collaborates with NVIDIA to Deliver an Enterprise-Class, Full-Stack GenAI Solution
- Voltron Data Launches Theseus to Unlock the Power of the Largest Data Sets for AI
- Amazon Aurora MySQL zero-ETL Integration with Amazon Redshift Now Generally Available
- Terra Quantum Announces Partnership with NVIDIA for Quantum-Enhanced Data Analytics
- AWS Announces 4 Zero-ETL Integrations to Make Data Access and Analysis Faster and Easier Across Data Stores
- AMD Instinct MI300 Series Launch: Accelerating Next-Gen AI and Supercomputing
- More This Just In…