Follow Datanami:
September 30, 2013

Syncsort Bolsters Mainframe-to-Hadoop Play with Circle Buy

Alex Woodie

Syncsort wants to become a single source of data integration tooling and technology for IBM mainframe customers who want to move mainframe data into Hadoop. The New Jersey company bolstered that goal with today’s acquisition of Circle Computer Group.

Organizations that run their business applications on IBM mainframes expect a certain level of quality and dependability in their IT systems. After all, if they could be assured that they could run their applications and store their data on an open source operating system running on inexpensive X64 servers–and do so in an effective and secure manner–they would do it in a heartbeat.

But there’s a reason that some of the world’s largest banks, hospital networks, and governments continue to write out those $10 million maintenance checks to Big Blue: the mainframe is extremely good at what it does. The mainframe’s momentum may have slowed, but there’s no sign that IBM’s original Big Iron platform is any closer to the grave than it was 20 years ago.

IBM mainframes still house a large amount of transactional data for the world’ largest institutions.

The advent of big data platforms like Hadoop offers tantalizing analytics opportunities for all organizations of a certain size, including the largest mainframe customers in the world. But when it comes to getting mainframe data into Hadoop, not just any old tool will do, according to Lonne Jaffe, CEO of Syncsort, which sells the DMX-h offering.

“When it comes to grown-up companies using Hadoop, not just the early adopters like the consumer Internet companies, it’s hard to suck in the data from the mainframe and some of these other legacy data warehouses in a way that high performance and secure and loads into all the nodes in the Hadoop cluster in parallel,” Jaffe says.

DMX-h addresses this need. “It’s really the only way to–in a high performance, secure way–get data off the mainframe into Hadoop to do big data analytics on it, and does all the transformations you need to do, like EBCDIC to ASCI and Cobol copybook imports and things like that,” he says.

Syncsort has already achieved some success with DMX-h. The company did the work with the open source community to uncouple the sorting process from the MapReduce paradigm. Its DMX-h engine “snaps into the MapReduce framework on every node in the cluster,” Jaffe says. This mainframe capability is unique in the industry, and has earned Syncsort partnerships with Cloudera and Hortonworks.

The acquisition of UK-based Circle Computer Group takes that mainframe story even further. As Jaffe explains, Circle’s middleware technology allows mainframe customers to move their data from expensive IMS or VSAM data repositories into more affordable and industry standard repositories, notably DB2/z, the version of DB2 built specifically for the System z mainframe. “It tricks the applications into thinking they’re accessing the data in IMS or VSAM, but actually they’re accessing the data in DB2/z,” he says. “It’s very high performance.”

This product fits nicely into Syncsort’s mainframe-to-Hadoop data integration strategy. The product today doesn’t support the capability to pull data directly out of IMS or VSAM into Hadoop today, but that’s at the very top of the product roadmap, Jaffe says. “There’s a bit of a talent acquisition here too,” he says. “We’re getting really good IMS and VSAM skills. If we want to be able to do that direct IMS or VSAM load into off-platform systems, we can do that.”

Many of Syncsort’s Hadoop customers today started using it as a platform to offload workloads from legacy data warehouses and some mainframe applications. As the Hadoop clusters become loaded with machine data or Web clickstream data, they realized they were missing out on the mainframe integration component, Jaffe says.

“At some point they said, ‘Wait a second. 80 percent of my corporate data is actually on the mainframe,'” Jaffe says. “How useful is it to have this Hadoop cluster doing advance big data analytics and I can’t access my actual data? All I can get to is my customer sentiment analysis and Web data.  If I want to get customer transactions or other systems of record type data, I need to access it.”

“We sort of stumbled into this market opportunity,” he continued. “It’s really quite remarkable that nobody is paying attention to it, just because of the staggering amount of data that’s still on the mainframe and how quickly some of these environments, like Hadoop, are growing. So we’re trying to move quickly so we can stay ahead of everyone on the space, and stay differentiated.”

Jaffe joined Syncsort several months ago after working for a time at CA Technologies, a big mainframe tool provider. He also spent 13 years at IBM, where he advised on roughly half of the IT giant’s software acquisitions over that time. He is not shy about stating his desire to grow through in-organic acquisition, as well as through organic growth.

“We’re really interested in companies–and Circle is a good example of this–that make us even harder to catch up with, in terms of this mainframe to Hadoop story line,” he says. “From an in-organic perspective, we’re looking in particular to buy companies that have powerful technology that we think we can have a lot of lift.”

Syncsort is looking to build on a position that’s already strong in the mainframe world. It has 4,000 mainframe customers, which represents more than half of all mainframes that exist in the wild. “The fact that we already run on half the world’s mainframes means that customers trust us more than they do some smaller open source based company,” Jaffe says. “If I’m a grownup company, alike a bank or a hospital or a counter terror organization, and I’m using Hadoop…there’s a bunch of things that I need to have in my software that [consumer oriented Web companies] don’t need. Some of it doesn’t exist yet, so we can build it, and some of it does exist, and those are really interesting acquisition targets for us.”

Syncsort is backed by a pair of private equity firms with about $4 billion in assets between them. It would be a shame if they got to enjoy this entire market all to themselves. But at this point, Syncsort would appear to have an early lead in a segment of the market that few companies are aware of.

Related Items:

Syncsort Aims to Bridge Hadoop ETL

Syncsort Contributes to Integration and ETL in Hadoop

The Three T’s of Hadoop: An Enterprise Big Data Pattern

Datanami