Language Flags

Translation Disclaimer

HPCwire Enterprise Tech HPCwire Japan
Leverage Big Data'14

January 12, 2012

The Evolving Art (and Business) of Data Curation

Over the course of 2012, one of the key phrases we can expect to start hearing a lot more often is data curation. While the term has been around for quite some time, particularly in academia and library science, it is finding a new home in enterprise contexts—and is attracting new attention in academia from funding agencies like the National Science Foundation.

For instance, the Johns Hopkins University Sheridan Libraries was awarded a $20 million NSF grant to build a “data research infrastructure” for managing the growing volumes of digitized research information. The five-year grant was one of two awarded by the NSF for what is being called “data curation.”

According to Sayeed Choudhury who directs the Digital Research and Curation Cetner and serves as associate dean at the university’s libraries, the impetus for the Data Conservancy was the need for large-scale digital data management in the science community. This is especially important, he says, because the application extend beyond these research communities, into the humanities and social sciences and beyond.

“Data curation is not an end but a means.” He says “Science and engineering research and education are increasingly data-intensive, which means that new management structures and technologies will be critical to accommodate the diversity, size and complexity of current and future data sets and streams.”

According the University of Illinois’ Graduate School of Library and Information Science, “Data curation is the active and on-going management of data through its lifecycle of interest and usefulness to scholarship, science and education.” The university further defines data curation, noting that it should “enable discovery and retrieval, maintain its quality, add value and provide for re-use over time.” According to their definition, this includes everything from authentication, archiving, management, preservation, retrieval and representation.”

The importance of data curation, especially over the coming years as organizations acquire and store increasingly diverse, mounting data sets, is being felt in the enterprise as well.

As Edward Curry and his team from the Digital Research Institute in Ireland noted, “With increased utilization of data within their operational and strategic processes, enterprises need to ensure data quality and accuracy…traditional approaches to curation are struggling with increased data voumes and near real-time demands for curated data.” He says that in response to these pressures, businesses have an ever-mounting set of motivations behind exploring data curation options.

Curry’s team points to Gartner’s figures that many organizations that are swimming in massive amounts of data have few policies in place to ensure the clarity, consistency and reliability of all of their data, in part due to lackluster data curation policies. In this era of big data and poor management of it, when then does it seem, on initial search, that there are no companies to be found offering pure-play data curation services on a consultancy basis?

It turns out there are plenty of companies with data curation possibilities—it’s just called something far different. Think in terms of data integration, data quality, data management—all of the phrases that tied into the major enterprise analytics platforms, but just not tied directly to the more academic-sounding concept of data curation.

Most of the major enterprise analytics and data management platforms offer the key characteristics of data curation as cited by the University of Illinois above. Information discovery, retrieval, quality checking, and usability over time are all parts of various all-in-one data management tools from companies, but for some reason, no vendors are tying their messaging around this important keyword phrase.

Consider companies like Informatica, Pentaho, Pervasive, IBM, SAS and others which offer platforms for all of the aspects of a sound data curation as cited above.In academia, projects like CAPS and the new curation effort at JHU are making headway by associating new solutions with the data curation movement.

Ina vendor landscape that hinges on the next big buzzword, the concept of data curation as the catchall term to replace the wider concept of “data management” seems fitting given the growing needs for more than just management of data—it requires a careful, even meticulous approach to defining the very environment and policies that management will work within.

Related Stories

Beyond Big Data: Addressing the Challenge of Big Applications

Pervasive Lends Supercomputing Center Analytical Might

New Techniques Turbo-Charge Data Mining

Share Options


» Subscribe to our weekly e-newsletter


There are 0 discussion items posted.


Most Read Features

Most Read News

Most Read This Just In

Sponsored Whitepapers

Planning Your Dashboard Project

02/01/2014 | iDashboards

Achieve your dashboard initiative goals by paving a path for success. A strategic plan helps you focus on the right key performance indicators and ensures your dashboards are effective. Learn how your organization can excel by planning out your dashboard project with our proven step-by-step process. This informational whitepaper will outline the benefits of well-thought dashboards, simplify the dashboard planning process, help avoid implementation challenges, and assist in a establishing a post deployment strategy.

Download this Whitepaper...

Slicing the Big Data Analytics Stack

11/26/2013 | HP, Mellanox, Revolution Analytics, SAS, Teradata

This special report provides an in-depth view into a series of technical tools and capabilities that are powering the next generation of big data analytics. Used properly, these tools provide increased insight, the possibility for new discoveries, and the ability to make quantitative decisions based on actual operational intelligence.

Download this Whitepaper...

View the White Paper Library

Sponsored Multimedia

Webinar: Powering Research with Knowledge Discovery & Data Mining (KDD)

Watch this webinar and learn how to develop “future-proof” advanced computing/storage technology solutions to easily manage large, shared compute resources and very large volumes of data. Focus on the research and the application results, not system and data management.

View Multimedia

Video: Using Eureqa to Uncover Mathematical Patterns Hidden in Your Data

Eureqa is like having an army of scientists working to unravel the fundamental equations hidden deep within your data. Eureqa’s algorithms identify what’s important and what’s not, enabling you to model, predict, and optimize what you care about like never before. Watch the video and learn how Eureqa can help you discover the hidden equations in your data.

View Multimedia

More Multimedia


Job Bank

Datanami Conferences Ad

Featured Events

May 5-11, 2014
Big Data Week Atlanta
Atlanta, GA
United States

May 29-30, 2014
St. Louis, MO
United States

June 10-12, 2014
Big Data Expo
New York, NY
United States

June 18-18, 2014
Women in Advanced Computing Summit (WiAC ’14)
Philadelphia, PA
United States

June 22-26, 2014

» View/Search Events

» Post an Event