In-Memory Data Grid Key to TIBCO’s Strategy

TIBCO is a bit of an anomaly in the big data world. While the software vendor is arguably one of the biggest players in the area, it doesn’t have a traditional database, which demonstrates its preference for processing data on the fly. But in the future, TIBCO’s in-memory technology, called ActiveSpaces, figures to play a more prominent role.

TIBCO is chasing the same big data opportunities that other vendors in the space are chasing. It wants to help customers capture the reams of data being generated from all types of sources, analyze it for patterns, and then generate some kind of actionable insight to help find or retain customers, to cut costs or increase revenue, and to avoid risk. TIBCO is not special in this regard, and shares a goal with big data’s new guard (those selling Hadoop, NoSQL, and related analysis tools) as well as BI’s old guard (those selling RDBMS and traditional data warehouses).

But TIBCO is going about this challenge in its own unique way, thanks to its history as The Information Bus COmpany and the success it has had selling its enterprise service bus (ESB) into some of the biggest and most demanding IT operations in the world, like those run by Proctor & Gamble, FedEx, and the New York Stock Exchange. TIBCO is big and mature enough not to get caught up in the big data hype, but still small enough to outmaneuver its bigger competitors, like IBM, when new opportunities arise.

“We’ve been handling big data since before big data was called big data,” Raj Verna, TIBCO’s vice president of sales for the Americas, said last week at TUCON. “We’ve had more time with big data than probably any company in our space.”

Big data is central to TIBCO’s strategy going forward, and it has either built or acquired several products to help it address its clients’ big data goals. The core strategy, as we discussed in a story last week, is to enable customers to identify patterns locked in the data, to codify those patterns into rules, and to automate the real-time execution of those rules against new data and opportunities as they arise.

Several of TIBCO’s products will play a role in this grand scheme, including the ESB, the Spotfire visualization tool, its StreamBase event processing engine, its ActiveMatrix business process management (BPM) software, and BusinessEvents, which provides complex event processing (CEP) capabilities.

Hadoop and NoSQL databases will also play a role in this, but not as part of the TIBCO stack. Instead, TIBCO relies on connectors to move information to and from big data stores, such as the Hadoop products from partners Cloudera and Hortonworks. TIBCO CTO Matt Quinn sees the partner approach working for the time being. “Quite frankly the space is moving so rapidly. If it’s not something we can see ourselves building or taking a distribution and building, then it’s much better to leverage somebody else’s R&D to do that work,” he said.

It appears that TIBCO’s ActiveSpaces in-memory technology, however, will eventually grow into something that more closely resembles what the broader market would consider a data store for big data analytics. Quinn identified ActiveSpaces as one of three critical technologies that all acquired products must be integrated with, the other two being Spotfire and the ESB itself.

“It’s pretty key,” Quinn says of ActiveSpaces, which was developed organically by TIBCO to address common customer problems, such as identifying fraud patterns. “ActiveSpaces was born out the early work around event processing. What would happen is we would have these event processing engines, and to be able to scale horizontally, you’d add more engines.  But you had to store the events somewhere, because sometimes the matching, the correlation, could take days or weeks.”

Today, TIBCO is building ActiveSpaces clusters that scale out to 200 nodes and can store as much as 10 TB of data. The technology often runs inside the memory of the application doing the processing, such as Spotfire, the ActiveMatrix BPM tool, the BusinessEvents CEP software, and its master data management (MDM) offering, which has become something of a data warehouse in its own right. StreamBase, which the company bought several months ago, will be tweaked to run in ActiveSpaces next.

TIBCO doesn’t insist on keeping all data in memory within ActiveSpaces, and it gives customers the option to save to disk for the purpose of disaster recovery. And while customers can choose to have data for their most commonly used applications stored on a local node, it also includes a shared-nothing architecture to protect from data loss.

While ActiveSpaces addresses some of the same problems as other databases, don’t call it a database. “General purpose databases, even the NoSQL guys–we’re all trying to circle around the same thing,” Quinn says. “We don’t like to call it a database because we don’t want people to say, ‘Oh I can just drop it in as a replacement for Oracle.’ That’s not what it was designed to do. It was really designed to solve a new class of problems. It’s the same language that Cassandra and Mongo and other people use, in exactly the same way, which is there’s a new class of application out there, and old databases don’t tend to work.”

Going in-memory with its apps makes sense for TIBCO, which has always had a penchant for speed. What’s different now is that TIBCO is being called upon to understand all types of other data, including the less structured stuff generated on the Web, which it hasn’t historically been able to support.

That’s about to change for TIBCO. At TUCON, the company announced a statement of direction to support more unstructured data in its products. “We are looking to see how we can store documents within the spaces, which would give us a moderate ability to process unstructured information,” Quinn said. “We know that side is a bit of a journey.”

Related Items:

Please Stop Chasing Yellow Elephants, TIBCO CTO Pleads

KPIs Get Personal with TIBCO Spotfire 6

TIBCO CTO Flashes R&D Future for Fast Data