Follow Datanami:
February 18, 2014

Hadoop No ‘Sideshow,’ Intel Says

Alex Woodie

When Intel launched its Hadoop distribution a year ago, there were some in the industry who viewed the move skeptically. Intel’s specialty, after all, is developing processors, motherboards, and systems. What does it know about writing and supporting open source software? Apparently, quite a bit. What’s more, the company’s commitment to open source software may be stronger than you think.

Intel used last week’s Strata conference in Santa Clara as a platform for launching the Intel Data Platform, a suite of big data analytics software built around Hadoop. The offering, which is still in beta, has the Intel Distribution for Apache Hadoop at its core, and also brings to the big data table various related Apache projects, including Spark, Shark, and Kafka.

Two versions of the Data Platform will be availed in the second quarter, including a free version and a premium version that carries technical support. The chip giant also launched its Analytics Toolkit (AT), which provides a new graph database developed by Intel, as well as classic machine learning algorithms.

The new software offerings are part of Intel’s effort to make Hadoop “a distributed operating system for big data,” Boyd Davis, vice president and general manager of Intel’s Datacenter Software division, said during a presentation at Strata. “Like other provider of Hadoop software, we’ve moved beyond the original Apache Hadoop project to incorporate other Apache projects that create a complete data platform.”

While Intel doesn’t have the Hadoop development chops of a Cloudera, Hortonworks, or MapR Technologies, the company has invested time and effort to make its Hadoop distro enterprise-ready. To that end, it’s bolstered the core open source technology with better management, security, and availability features.

From the sound of it, Intel is ready to take a more active role in the development of Hadoop itself, with the goal of ensuring that innovation around Hadoop remains in the open source realm. It may sound odd to think of Intel as Hadoop’s open-source defender, but apparently, that is the role that it is taking on.

“I believe the Hadoop market is at a crossroads,” Davis said. “There’s the potential for the market to fragment into islands of proprietary technologies that are based on the open source project but in fact, to be usable, require a closed, expensive ecosystem. Or we have the option to make it move forward in a Linux-like fashion, where it’s open, multivendor, and creates a platform for innovation and drives rapid growth.”

Davis says the Hadoop market today is where the Linux market was 15 years ago. At the time, Unix operating systems running on proprietary RISC platforms dominated the high-end server market. Then along comes Linus Torvald’s open source platform, which at first nobody paid much attention to. But before long, Torvald’s adorable little penguin had gobbled up scads of market share from the dominant server vendors.

My, how times have changed. Intel’s X86/64 processor architecture dominates the entire server business–high-end, midrange, and low-end. Its chips are used in most HPC installations, too. If Hadoop is going to be “the next big thing” in computing and live up to the hype (which is not a foregone conclusion), then Intel wants to be fully involved. It already has a leg up on the market, because nearly all Hadoop implementations run Intel’s commodity gear. IBM supports Hadoop running on Linux and Power processors, which are based on the RISC architecture, but there are few reports of customers.

Intel’s interest and participation in the Hadoop community can be viewed in this light. “This is why Intel is in this market,” Davis says. “This is not a sideshow for us. Growth is a critical agenda for our business, and this is a critical part our strategy to grow our $12 billion core datacenter business.”

Related Items:

Intel Goes Graph with Hadoop Distro

Is Intel Raining on the Hadoop Distro Startup Parade?

Rethinking Real-Time Hadoop

Datanami