Follow Datanami:
November 21, 2012

MIT Sloan Sees Big Future in Big Data

Ian Armas Foster

Last week’s Supercomputing conference had a larger focus on big data than SCs past, with HPCwire’s Michael Feldman and Intersect360’s Addison Snell declaring “big data” as one of the winners of the conference in their Soundbite podcast.

The MIT Sloan Management Review, which runs its own well-known thought leadership series, took a gander at the big data industry in an attempt to determine and communicate where precisely it is heading and how companies can take advantage.

According to the report, there are three aspects in enterprise that would spell success for an organization looking to run big data: paying attention to flows instead of stocks, relying on data scientists instead of data analysts, and moving analytics from the pure IT perspective to a broader operational function.

The first point, moving one’s attention from stocks to flows, refers not to things traded on Wall Street but rather building up warehouses or stocks of data and analyzing it at one time. As data comes in almost constantly, it becomes necessary to continually analyze that data. But an unfortunate side effect is that the one who looks at the past three months’ worth of data ends up being stuck in the past.

Marketing campaigns are getting more personalized and targeted as big data allows companies to understand their customers better. The report’s first point of viewing data flow as a stream instead of a stack feeds nicely into the second point of moving from data analysts to data scientists. Insteadof data analysts asking simple questions of small datasets, data scientists are allowed to model complex queries from the large databases. It is this talent and research that allows marketing specialists to better target their campaigns.

Of course, the relative shortage of data scientists has been well-documented on this site and others over the last year or so. The report mentions EMC Greenplum’s efforts to work with universities to train the data scientists of tomorrow, but there is likely to be a gap in talent for the foreseeable future.

To help bridge that gap, the report mentions its third point: incorporating big data analytics into all aspects of an organization instead of harboring it in the IT section and copying it out when necessary. In particular, they mention eBay’s problem of having data copied 20 to 50 times over the organization, which is a waste of valuable resources. Integrating the entire operation into one data center, whether it be in the cloud, on a Hadoop cluster, or wherever, eliminates that inefficiency.

For the most part, these trends seem relatively straightforward to those in the know. But those in the know may not necessarily be those who are making the executive decisions. This MIT Sloan report is a helpful guide to that end.

Related Articles

20 Lessons Enterprise CIOs Can Learn from Supercomputing

MIT Sloan Event Unravels Data Trends

CSAIL Rides Big Data Visualization Wave

Datanami