Follow Datanami:
May 27, 2020

Lucidworks Announces Advanced Linguistics Package

SAN FRANCISCO, Calif., May 27, 2020 — Lucidworks announced the Advanced Linguistics Package for Lucidworks Fusion to power personalized search for users in Asian, European, and Middle Eastern markets. Lucidworks now embeds text analytics from Basis Technology, the leading provider of AI for natural language processing. With the Advanced Linguistics Package, global organizations that support multiple languages can make the information and insights they manage more accessible, more relevant and more personalized for their global audience.

In order to provide a personal digital experience for all users, global organizations need a search application that can understand the text from a dozen or more languages. Building, testing, and maintaining the many algorithms and models required to properly support each language is challenging and expensive. Asian, Middle Eastern, and certain European languages require additional processes to handle unique linguistic phenomena, such as lack of whitespace, compound words, and multiple forms of the same word.

“When it comes to applying AI for understanding multiple languages, Basis Technology is superior,” says Radu Miclaus, Lucidworks Director of Product, AI and Cloud. “When we match that capability with the best-in-class AI-powered search platform of Lucidworks Fusion, we expect great accuracy and performance enhancements in information retrieval for the digital experience. Basis Technology has worked with the world’s largest companies to make their global search work brilliantly for over 20 years. Rosette, their text analytics product, is deployed in many of the largest government and business applications. We are very excited to formalize this partnership at the product level and continue to drive value for our customers together.”

Lucidworks’ Advanced Linguistics Package provides language processing in more than 30 languages and advanced entity extraction in 21 languages. By accurately analyzing the text, in the language it was written, Rosette helps the Lucidworks Fusion platform deliver the right answers to every user, regardless of where they work or what language they use.

“This is a story of two complementary technologies growing even closer together. Search and natural language processing are essential ingredients for making raw data useful,” explains Steve Cohen, Basis Technology, COO. “Whether you are building a more personalized user experience, a smarter product recommendation, or simply trying to answer a question, having AI-driven search with tested, proven NLP will improve efficiency and allow you to get the most possible value from your data.”

More details on Advanced Linguistics Package for Lucidworks Fusion below:

Named Entity Recognition – Rosette extracts and structures key information in multilingual text by automatically identifying names, places, organizations, products, and key phrases in 21 languages. With Rosette, customers can increase search relevance by filtering results and returning ones containing the entities most pertinent to your search.

Tokenization – For some languages such as Chinese, Japanese, and Korean, there are not always spaces in between characters or words. Unlike in English, any one character could be a single word, but typically is not, making it difficult for machines to analyze. Tokenization identifies the words so that the search engine can deliver relevant results instead of substring matches.

Language Identification – The Advanced Linguistics Packages also includes identification for 55 languages.

Interested in learning more about how Fusion Advanced Linguistics Package powers a personalized digital experience for some of the world’s largest global brands? Visit us at

About Lucidworks

Lucidworks builds AI-powered search solutions for many of the world’s largest brands. Fusion, Lucidworks’ advanced development platform, provides the enterprise-grade capabilities needed to design, develop, and deploy intelligent search applications at any scale. Companies across all industries, from consumer retail and healthcare to insurance and financial services, rely on Lucidworks every day to power their consumer-facing and enterprise search apps. Lucidworks’ investors include Francisco Partners, TPG Sixth Street Partners, Top Tier Capital Partners, Shasta Ventures, Granite Ventures, and Walden International. Learn more at

Source: Lucidworks