Patronus AI Raises $17M to Detect LLM Mistakes at Scale

SAN FRANCISCO, May 23, 2024 — Patronus AI has announced it has raised a $17 million Series A round, bringing the total amount raised to $20 million. The financing was led by Glenn Solomon at Notable Capital with participation from Lightspeed Venture Partners, Datadog, Gokul Rajaram, Factorial Capital, and several leading software and AI executives.

Founded by machine learning experts from Meta, Anand Kannappan and Rebecca Qian, Patronus AI is the first automated evaluation and security platform that helps companies detect large language model (LLM) mistakes at scale. Using proprietary AI, the platform enables enterprise development teams to score model performance, generate adversarial test cases, benchmark models and more. Patronus AI automates and scales the manual and costly model evaluation methods prevalent in the enterprise today, enabling organizations to confidently deploy LLMs while minimizing the risk of model failures and misaligned outputs.

“Model hallucinations and safety risks are here to stay,” Anand Kannappan, CEO, Patronus AI. “What enterprises need is transparency into model performance and accuracy in order to circumvent risks. For the first time, we’re giving companies a way to truly understand what they are working with so they can deploy LLMs with confidence. It’s the only path to innovation.”

Since the company’s official launch in late 2023, the company has released industry research shedding light on the pervasive accuracy and safety problems inherent with LLM use. The first is FinanceBench, the industry’s first benchmark for LLM performance on financial questions. The research revealed that popular models, like OpenAI’s GPT-4-Turbo, only got 19 percent of answers right when reading an entire SEC filing. Patronus AI also released CopyrightCatcher, the first Copyright Detection API for LLMs, showing that LLMs are capable of plagiarizing content without the proper guardrails.

In addition, a number of large Fortune 500 companies already use Patronus AI to implement their generative AI initiatives safely within their organizations. Patronus AI plans to use the additional capital to scale its AI research, engineering, and sales teams, train evaluation models, and develop new industry standard benchmarks.

“The intense interest among enterprises to bring LLM-based applications to market makes this one of the fastest growing markets we’ve ever seen at Notable Capital. Despite this swelling demand, the persistent issues of accuracy and safety in large language models pose risks that large enterprises cannot afford to take,” said Glenn Solomon, Managing Partner, Notable Capital. “Patronus AI is not only tackling this issue with their industry-first platform, their groundbreaking research is improving the reliability of AI outputs and setting new industry benchmarks for responsible AI use. Anand and Rebecca’s background in AI research are precisely what the industry needs at this critical juncture, and we’re thrilled to help them scale their solutions to any organization in need.”

About Patronus AI

Patronus AI is the first automated evaluation and security platform that helps companies use large language models (LLMs) safely. For more information, visit https://www.patronus.ai or reach out to [email protected].

Source: Patronus AI