Follow Datanami:
May 12, 2015

Wikibon Analyst Kelly: Big Data is Entrenched

Back in 2011, when the respected Wikibon researcher Jeff Kelly began tracking big data, the technology was, in his view, “long on promise but short on specifics.”

In his final blog post for the Wikibon Project, Kelly concludes that the technology has come full circle, asserting that its steady application has helped to solve real-world business challenges while evolving into a “competitive differentiator” in a world gone digital. It has even helped deliver sports championships.

“Big data is no longer just promise, but reality,” he asserts.

What’s the evidence? For one thing, an exploding adoption rate over the last four years that has boosted the big data hardware, software and professional services market to more than $27 billion, according to Wikibon. The market stood at about $7 billion when Kelly started tracking it for Wikibon four years ago.

The market tracker forecast in April that the big data market would top $60 billion by the end of the decade.

Another indication is the growing number of big data players and startups ranging from Hadoop distribution vendors to analytics specialists. Big data has also become a standard feature among cloud providers and networking companies seeking to push intelligence as close to data as possible.

Thanks to a creative open source community of developers, Kelly notes that “finicky” frameworks like Hadoop have “evolved into a much more comprehensive, enterprise-grade, multi-application supporting big data platform.”

Kelly also argues that big data has helped transformed entire industries—see AirBnB, Facebook, Uber—as well as some of the world’s largest business and financial institutions. The adoption rates for big data and data analytics at these companies is so fast that most cannot find enough data scientists to sift through the data waterfall. Many are now investing in graduate-level programs to train the next generation of data scientists and big data practitioners, the core of the community Wikibon serves.

Another good example of the impact of big data is professional sports. Perhaps the most data-driven of big-time sports is Major League Baseball. Before the Money Ball phenomenon of the early 2000s was the Sabermetrics craze of the 1980s that applied empirical data analysis to baseball stats. Now it’s all about big data. Kelly cited the recent success of the World Champion San Francisco Giants who have embraced data analytics on and off the field. The result: The team has won three of the last five World Series.

In an interview with Giants’ CIO Bill Schlough, Kelly highlighted how the ball club is leveraging big data at its ballpark, AT&T Park. Data applications range from pouring through players statistics to discerning trends that could provide an edge in a close playoff game to using sensor data to improve stadium operations.

“We really feel like there’s an opportunity to capture some of the data that has never been captured before,” Schlough told Kelly. The team is, for example, using cameras to capture billions of data points about the movement of players and the baseball on the field. “Every team is going to be there in the next few years, but it’s something we have been experimenting with for three or four years.”

Still, Schlough added, echoing a point made frequently by big data analysts like Kelly, “There’s no such thing as a sustainable competitive advantage to us.”

Recent items:

Booming Big Data Market Headed for $60B

Grad Schools Move to Fill Data Skills Gap

Datanami