Language Flags

Translation Disclaimer

HPCwire Enterprise Tech HPCwire Japan


February 05, 2013

Weighing MapReduce Against Parallel DBMS


At the crux of many an enterprise-scale big data system stands either MapReduce or a parallel database management system. But which is more efficient?

Researchers from Dublin Institute of Technology, the University of Lisbon, and the National University of Ireland Galway sought to answer that question when they studied the direct comparisons between MapReduce and parallel DBMS.

The team argues that while plenty of research has been published on the topic, the results of those studies represent conflicting and inconsistent viewpoints. “Some presented MapReduce as more flexible and capable of producing better performance,” the team noted. “Others have presented the opposite opinion. Finally, some suggested that the two approaches can be used together.”

Further, none of the studies referenced in the article were published in the last two years, indicating that the time is ripe to re-examine the issue.

One of the biggest flaws with the parallel DBMS is that it is ill-suited to deal with a single node failure. As a result, a big query or job running on parallel DBMS will have to intermittently stop and store everything such that it can be re-started upon node failure. Meanwhile, MapReduce can simply shift jobs to other nodes if one fails. “As more nodes are added into the parallel DBMS environment, the chance of a node failure increases. Parallel DBMS do not handle node failure. MapReduce has been designed to run on thousands of nodes and is inherently fault tolerant.”

Parallel DBMS is also limited by its SQL-based roots. While the Structured Query Language serves as an efficient method for writing queries on relational data, it struggles with unstructured text or social media data. By assigning value to characters or words of text in the mapping stage, MapReduce is designed to overcome this difficulty. Also, since MapReduce is not limited by the bounds of the SQL language, more complex questions can be asked of the data.

As such, multiple-step queries that foster more complex analytics are better suited for MapReduce. “The aggregation in SQL is not able to process these multiple step data flows. Performing these tasks in many steps reduces the performance benefits gained from parallel DBMS.”

Finally, since MapReduce is an open source technology, it can be implemented with relatively low cost compared to parallel DBMS, which the researchers labeled an “expensive investment.”

Of course, there were some advantages to parallel DBMS, otherwise the discussion would not even take place. In fact, one of MapReduce’s edges—the lack of SQL limitation—is also a significant drawback, as the average skill level of someone using MapReduce must exceed that of one who uses SQL. Many organizations do not have the talent at their disposal to deal in complex MapReduce queries.

Also, parallel DBMS sports a better capability for repeated structured data processing. Once a large dataset makes it into the DBMS, it can be queried more repeatedly and efficiently than if one were using MapReduce. “Parallel DBMSs are excellent at efficient querying of large data sets. Conversely, MapReduce is much slower by comparison, but is significantly better at complex analytics and ETL tasks.”

There is opportunity, the researchers noted, for the two systems to work in tandem. Parallel DBMS can suffer from an I/O bottleneck when large amounts of data are input. If the data is structured and iterative queries need to be run on it, it would be advantageous to get that data into the DBMS. MapReduce could help compress the data before transferal. “MapReduce can be useful for one-time queries while parallel DBMS can be useful for repeated queries. This property of MapReduce makes it ideal for transforming data prior to load into the Parallel DBMS.”

MapReduce has parallel DBMS beat in I/O efficiency, cost, surviving node failure, ability to handle unstructured data, and complex analytics capabilities. Meanwhile, parallel DBMS takes the cake when it comes to ease of use and repeated queries of structured data.

As it stands now, whether to rely on one or the other is dependent on the organization’s resources and application requirements. However, the two should necessarily be considered to be at odds with each other, as likely some form of a combination of the two is more likely to produce a comprehensive, all-data management system.

Related Articles

eScience and a Tale of Two Cities

Building the Library of Twitter

New Ropes for Scaling the SQL Wall

Share Options


Subscribe

» Subscribe to our weekly e-newsletter


Discussion

There are 0 discussion items posted.

 

Most Read Features

Most Read News

Most Read This Just In



Sponsored Whitepapers

Planning Your Dashboard Project

02/01/2014 | iDashboards

Achieve your dashboard initiative goals by paving a path for success. A strategic plan helps you focus on the right key performance indicators and ensures your dashboards are effective. Learn how your organization can excel by planning out your dashboard project with our proven step-by-step process. This informational whitepaper will outline the benefits of well-thought dashboards, simplify the dashboard planning process, help avoid implementation challenges, and assist in a establishing a post deployment strategy.

Download this Whitepaper...

Slicing the Big Data Analytics Stack

11/26/2013 | HP, Mellanox, Revolution Analytics, SAS, Teradata

This special report provides an in-depth view into a series of technical tools and capabilities that are powering the next generation of big data analytics. Used properly, these tools provide increased insight, the possibility for new discoveries, and the ability to make quantitative decisions based on actual operational intelligence.

Download this Whitepaper...

View the White Paper Library

Sponsored Multimedia

Webinar: Powering Research with Knowledge Discovery & Data Mining (KDD)

Watch this webinar and learn how to develop “future-proof” advanced computing/storage technology solutions to easily manage large, shared compute resources and very large volumes of data. Focus on the research and the application results, not system and data management.

View Multimedia

Video: Using Eureqa to Uncover Mathematical Patterns Hidden in Your Data

Eureqa is like having an army of scientists working to unravel the fundamental equations hidden deep within your data. Eureqa’s algorithms identify what’s important and what’s not, enabling you to model, predict, and optimize what you care about like never before. Watch the video and learn how Eureqa can help you discover the hidden equations in your data.

View Multimedia

More Multimedia

ISC'14

Job Bank

Datanami Conferences Ad

Featured Events

May 5-11, 2014
Big Data Week Atlanta
Atlanta, GA
United States

May 29-30, 2014
StampedeCon
St. Louis, MO
United States

June 10-12, 2014
Big Data Expo
New York, NY
United States

June 18-18, 2014
Women in Advanced Computing Summit (WiAC ’14)
Philadelphia, PA
United States

June 22-26, 2014
ISC'14
Leipzig
Germany

» View/Search Events

» Post an Event