Has Big Data Peaked?
Fewer companies are planning to invest in big data projects over the next two years due to concerns about getting a return on investment (ROI) and the difficulty in moving big data projects from pilot to production, among other reasons, Gartner said today.
According to Gartner‘s online survey, which involved 199 Gartner research clients questioned in June, the percentage of organizations planning to invest in big data within the next two years fell from 31% when Gartner asked in 2015 to 25% this year.
That grim forward-looking prediction comes on the heels of strong growth up to this point. The analyst group’s survey shows that 48% of companies have already invested as of June 2016. That’s up 3% compared to 2015.
Overall, nearly three-quarters of the survey respondents say they either have already invested in big data projects or are planning to invest. That figure meshes well with the perception, widely held in the industry, that advanced analytics based on big data has become a necessity for businesses to succeed going forward.
However, the reality on the ground is more nuanced than that, according to Gartner research director Nick Heudecker. “Investment in big data is up,” the influential analyst said in a statement, “but the survey is showing signs of slowing growth with fewer companies having a future intent to invest.”
Heudecker attributes some of that slowdown to the difficulty in getting a “tangible [ROI] that can be determined upfront.” This is a clear reference to the type of open-ended data science project that starts with the notion of collecting all data, and then figuring out how to make use of it later.
The ROI question has become a major sticking point for big data. While companies have powerful new tools and techniques for analyzing new data types, they’ve found it harder to get concrete business value from the insights they’re generating than they first imagined.
Teradata (NYSE: TDC) analyzed the ROI question in a study last year, and found that that, while about two-thirds of companies are getting value from big data investments – on the order of a 1% to 3% boost — it’s not easy going by any means.
“It’s not just, ‘Let’s go buy some technology deploy it and it’s great,'” Chris Twogood, Teradata’s Vice President of Product and Services Marketing, told Datanami a year ago. “It’s about being able to build data into the DNA of the company, which means you have to deal with people and process and technology, and build a culture to adopt it.”
Another hurdle that Gartner spotted is the difficulty in getting big data projects out of the pilot stage and into production. Of the three-quarters of survey respondents who have already invested in big data (or are planning to), only 15% of them have deploying big data projects to production, according to Gartner. That’s up about 1% from last year.
One possible cause of that, Heudecker says, is that big data projects are running up against stiff competition for funding in tight IT budgets. According to Gartner, only 11% of survey respondents from companies that have already invested in big data reported that their big data investments were as important, or more important, than other IT initiatives. Nearly half stated that their big data investments were less important than other IT initiatives.
But it wasn’t all doom and gloom in the big data department in Stamford, Connecticut. Just as Gartner nixed big data off the Hype Cycle for Emerging Technology in 2015 because, it said, big data had spread everywhere, one shouldn’t be surprised to see a slowdown in projects defined strictly by the phrase “big data.”
“Another reason [for the slowdown in production big data projects] could be that the big data initiative is a part of a larger funded initiative,” Heudecker says. “This will become more common as the term ‘big data’ fades away, and dealing with larger datasets and multiple data types continues to be the norm.”
Clearly, the big data phenomenon isn’t over. The size of data, its speed and importance, isn’t shrinking. But it’s also clear that “big data” as a single entity doesn’t carry the same influence as it once did. That’s not necessarily a bad thing.