Under New CEO, Lucidworks Aims to Redefine Search and Itself

Lucidworks today unveiled an ambitious new enterprise search application called Fusion that uses advanced signal processing and analytics to drive a new level of personalization and push-delivery of information to users. The product is the culmination of a year-long development initiative under new CEO Will Hayes, who aims to carve a new big data niche for the Solr backer.

Search is a ubiquitous component of the Internet, the original “killer app.” Type in your query, hit enter, and voila: a million hits are delivered right to your face. This has been the foundational business model of the Web ever since Netscape gave us the first browser and Yahoo provided us that first Google-based search experience, oh so many years ago.

But what if the search action didn’t involve typing into a box, but instead involved could be triggered in other ways, such as where you are, what’s on your calendar, or what’s trending on Twitter? What if the search engine could learn about who you are, what your interests are, and shape the search results accordingly?

These are the capabilities that Lucidworks is delivering with Fusion. The new offering sits on top of the open source Solr engine–which is already included in almost every distribution of Hadoop and many other enterprise applications–and provides a platform for building a new class of highly personalized big data applications, Hayes says.

“What’s interesting about Fusion is it’s not just about that retrieval of data at scale, but the context we can provide on top of the data,” Hayes tells Datanami. “It’s the fact that search engines are aware of who you are, your history, what devices you’re on, your geo-location. It’s the ability to create a contextual data experience for your users.”

In a sense, Lucidworks is aiming to turn the concept of search on its head, and central to that plan is Fusion’s new signal processing capabilities. The signal is anything in the world that’s relevant to a user and his data. In an ecommerce setting, it could be as simple as a mouse click. In the context of a distributor who’s in the field visiting customers, the signal may come from the GPS on the smartphone. In a manufacturing setting, it could be an ERP system signaling that that inventory of a particular component is getting low. An investment bank may capture signals from the stock tickers, while a pharmaceutical company may set up signals to watch for new research in a particular drug, side effects, or molecules.

“All of these signals get incorporated and then through machine learning algorithms and analytics, we’re able to surface better sorting algorithms for your data,” Hayes says. “It’s beyond the indexing and query capability [of Solr and Lucene]. It’s really about intelligent processing of data, understanding context, and being able to find relevant information based on different signals. That’s where this data-driven paradigm comes from. It’s understanding your context and your intention and retrieving and exposing data that’s relevant to you based on those things.”

If you use Apple’s Siri, regularly shop on Amazon, or watch videos on Netflix, you’re familiar with the power of contextualized search functions. These Web giants have thousands of data scientists working with advanced analytic tools and machine learning algorithms to create highly personalized Web experiences for their users. As consumers use these services in their private lives, they’ve come to expect that level of personalization in the applications they use at work. But so far, enterprises have been slow to deliver them, Hayes says.

Hence the business opportunity for Hayes and his team. “Lucidworks is the first company to incorporate a platform to start to create these experiences for enterprise, where they don’t have to have an entire team of data scientists, like Google does,” Hayes says. “We’re all familiar with what context means in an e-commerce shopping experience. But what we think is most exciting and where we’re headed with our technology and vision is how enterprise applications can behave like these consumer applications to bring more of that data experience to the end user, be it looking at customer support records, call detail records, or going into these massive data stores like Hadoop to not only make it accessible, but put context around it.”

Fusion is not open source, and you won’t see it as part of your Hadoop distribution. But the new application is definitely geared toward Hadoop and the massive amounts of information that organizations are starting to store there. The NoSQL-based Solr search engine is already heavily embedded within the big yellow elephant, and Lucidworks is banking that that presence greases the pathway toward mass Fusion adoption.

Lucidworks CEO Will Hayes

If the prospect of incorporating the Internet of Things (IoT) into your enterprise sounds daunting, that’s because it is. We’re at the beginning of a new paradigm in how we use and think about data. But Lucidworks aims to make it easy for customers to get started with Fusion by providing a number of templates. Hayes says one early adopter who didn’t have enterprise search experience was able to use Fusion to create a consumer Web experience from an existing product catalog in five days. “We’re making this really simple for you,” he says. “That’s the goal of this platform, to make these advanced capabilities far more attainable for the enterprise.”

Hayes has been CEO of Lucidworks for only three months (his new position was made public today). But you could argue that the former Splunk engineer and executive is already having a transformative effect on the company. “I came from a very strict product culture, we were obviously very successful at Splunk and we wanted to bring some of that DNA back into Lucidworks,” he says. “That’s really our primary goal, that we build and go to market with the strongest technology.” He certainly seems to be off to a good start.

Related Items:

Splunk Gets ‘Lucid’ About Solr-Powered Search

It’s Sink or Swim in the IoT’s Ocean of Bigger Data

Searching Big Data’s Open Source Roots