Language Flags

Translation Disclaimer

HPCwire Enterprise Tech HPCwire Japan
Leverage Big Data'14

February 04, 2013

A Holistic Approach to Big Data Analytics

The race to build an effective holistic big data analytics system is a lucrative one being run by several Hadoop-focused startup companies, including Karmasphere and Datameer.

Manish Jiadani, Karmasphere’s Senior Director of Product Management, hosted a talk where he discussed Karmasphere’s new Hadoop-based data analytics offering, called Karmasphere 2.0.

For Karmasphere, a company founded in 2005, there exist five pillars that hold up a worthwhile analytics system. Some of those pillars are fairly common across vendors, including the ability to garner ‘actionable insight’ along with hosting ‘unconstrained analytics.’

Their first factor, collaboration, is also fairly common, but it carries another level of significance for Karmasphere. They hope to ease the collaborative process by automatically presenting the data users wish to work with graphically when it comes out of Hive.

The Karmasphere system works in four steps. The first step, which they call ‘ingest,’ takes in data in “wizard-based data ingestion.” According to Jiadani, the data is immediately visualized in a tabular manner before reaching the user in the exploration step. Meanwhile, the metadata is poured into Hive tables.

The second step would allow end users to ‘explore’ the data through “visual, iterative data exploration with filtering and sorting wizards.” Again, effective initial visualization is the key here. Instead of tabulating the information through spreadsheets like Datameer, the interface represents the data graphically, with the emphasis being on fostering a collaborative environment.

The user would then analyze the data through iterative SQL analysis before connecting to a visualization tool like Tableau and Spotfire to represent results.

These features are represented in the figure below, which details the architecture underlying the Karmasphere platform.

One of the challenges in building a comprehensive Hadoop-based analytics system is finding a way to let business and end users efficiently work with the data. As shown by the diagram, Karmasphere would run atop Hadoop, sending and receiving its data from Hive. Upon arrival, the system analyzes the data through several means, including through SAS and SPSS analytics as well as a custom text analytics application from their analytic asset repository.

From there, according to Jiadani, the data can be output to a NoSQL database, back into Hive, or into, as mentioned before, a visualization package.

It will be interesting to see how the platform stacks up with that of competitor Datameer, who also think of themselves as a do-it-all big data analytics company. Whether or not Hive, Hadoop’s data warehousing tool, can serve as the principal feeder and keeper of Karmasphere’s data as opposed to a more private option like EMC Greenplum or HP Vertica will be an essential factor in the success of the Karmasphere system.

Indeed, Karmasphere CEO Gail Ennis spoke last year on the limitations of Hive but noted that its effectiveness is rapidly growing. She argued back in June that “the constraints are going to be addressed very soon because Hive adoption and community development work is moving quickly.”

Ennis and her company will hope that Hive maintains its development pace and that their system will run smoothly from it.

Related Articles

New Ropes for Scaling the SQL Wall

RHadoop Brings Code to Data

Gnip's Take on the Library of Twitter

Share Options


» Subscribe to our weekly e-newsletter


There are 0 discussion items posted.


Most Read Features

Most Read News

Most Read This Just In

Cray Supercomputer

Sponsored Whitepapers

Planning Your Dashboard Project

02/01/2014 | iDashboards

Achieve your dashboard initiative goals by paving a path for success. A strategic plan helps you focus on the right key performance indicators and ensures your dashboards are effective. Learn how your organization can excel by planning out your dashboard project with our proven step-by-step process. This informational whitepaper will outline the benefits of well-thought dashboards, simplify the dashboard planning process, help avoid implementation challenges, and assist in a establishing a post deployment strategy.

Download this Whitepaper...

Slicing the Big Data Analytics Stack

11/26/2013 | HP, Mellanox, Revolution Analytics, SAS, Teradata

This special report provides an in-depth view into a series of technical tools and capabilities that are powering the next generation of big data analytics. Used properly, these tools provide increased insight, the possibility for new discoveries, and the ability to make quantitative decisions based on actual operational intelligence.

Download this Whitepaper...

View the White Paper Library

Sponsored Multimedia

Webinar: Powering Research with Knowledge Discovery & Data Mining (KDD)

Watch this webinar and learn how to develop “future-proof” advanced computing/storage technology solutions to easily manage large, shared compute resources and very large volumes of data. Focus on the research and the application results, not system and data management.

View Multimedia

Video: Using Eureqa to Uncover Mathematical Patterns Hidden in Your Data

Eureqa is like having an army of scientists working to unravel the fundamental equations hidden deep within your data. Eureqa’s algorithms identify what’s important and what’s not, enabling you to model, predict, and optimize what you care about like never before. Watch the video and learn how Eureqa can help you discover the hidden equations in your data.

View Multimedia

More Multimedia


Job Bank

Datanami Conferences Ad

Featured Events

May 5-11, 2014
Big Data Week Atlanta
Atlanta, GA
United States

May 29-30, 2014
St. Louis, MO
United States

June 10-12, 2014
Big Data Expo
New York, NY
United States

June 18-18, 2014
Women in Advanced Computing Summit (WiAC ’14)
Philadelphia, PA
United States

June 22-26, 2014

» View/Search Events

» Post an Event