What Exactly Is Big Data, If It’s Neither About Big Nor Data?

Many people ask us why there doesn’t seem to be an accepted definition for Big Data in view of the massive press it receives. And given the amount of marketing effort expended by the global IT vendors who are targeting this bandwagon (and the venture-funded start-ups) they can be forgiven for being confused.

Just as most consumer marketing – curiously – is focused on adolescent buyers with a fraction of the purchasing power of their parents and grandparents, so it is with Big Data and its promoters. They shower their traditional customers – the IT departments – with buzzwords, while failing to engage with (or indeed, completely alienating) the business managers who have both the business problems that analytics can solve and direct control over the budgets.

Why is this, and what does it mean for Big Data adoption? My personal view is that the targeting of IT departments stems from both laziness and geekiness. Laziness, in the sense that – with few exceptions – these large vendors got to be large without needing to address directly the concerns of business managers – the IT departments they deal with have until now performed this translation task for them. Geekiness, in the sense that the smaller specialist vendors are often managed by technologists, and so their market messages – for reasons of background – also tend to focus on “speeds and feeds” rather than on solving practical business problems.

This has negative consequences for Big Data take up and risks becoming the sand that makes “The New Oil” less attractive. The massive hype – the promise of Big Data – that has appeared in the press has served to both raise expectations among business managers, and also to confuse them. Vendors hoping that the IT department will take the lead in initiating Big Data projects are deluding themselves.

IT departments today are:

• A) far too busy with day to day work (BYOD, mobile access, security, etc) to do this,
• B) too distant from the concerns of functional managers to be able to guide them in selecting promising areas for piloting analytics, and
• C) often unskilled as data scientists.

So let us return to the question of what the term “Big Data” actually means. Our experience in working as Management Consultants in Big Data with clients in retail, ecommerce, supply chain, insurance and other industries (as well as with investors and start-ups) suggests that there is no accepted definition; however we have been able to evolve and validate our own definition of what it’s not and what it is.

Fig. 1

Let’s take each of these in turn. What does “Answering the questions you have yet to ask” mean? Well, first it means collecting a wider range of data than your current or traditional analysis requires. So instead of running the same monthly reports on your financial system, for example, or on your website stats, you incorporate data that may have no currently known relationship with how you analyze your data today.

Second, it means deriving new insights by combining this disparate data (that may also be external, such as geospatial or social) in new ways. This is more about “data discovery” than about “weekly reporting packages”.

Fig. 2

The third part is about more effective visualization – helping stakeholders absorb, share and exploit insights from new data analyses. If the first pillar (“Question”) is about helping managers better understand the drivers of their business, the second (”Accelerate”) is about speed and accuracy of business response. Once the discovery part has yielded some insight – the second aspect of big data is about putting it to work. This might be via ondemand reports that enable managers to make decisions more rapidly or it might be the codification of a set of rules that are applied to the incoming data to make decisions automatically (aka “algorithm”).

We have identified three key areas of faster business response:

• Personalising (Customer experience, “single view of the customer”)
• Optimising and predicting (eg Customer service, supply chain)
• Identifying patterns in new ways (such as crisis/fraud detection and response)

The third pillar in our definition of Big Data is “Transform”, the process by which it is deployed. Many vendors see Big Data as a “technology wave” but we see it as more of a “business transformation wave”. This is precisely the reason why we do not see mainstream adoption until the business change issues are properly addressed. Unlike traditional IT-centric projects we see the adoption of Big Data as: having three key requirements:

• Small scale, incremental, “ Trial and Error” (see Fig, 3) – agile rather than “Big Bang”
• Cuts across departments – in terms of processes, reporting lines and data
• Requires an organisation-wide culture change to “data-driven”

Fig. 3

It points above is the need to learn to exploit the data opportunity by experimentation (see Fig 3). Related to this is the principle of “Data as a Corporate Asset” and by extension the level of “Analytical Maturity” that each business has achieved. The good news is that as more businesses deploy, there is increasing agreement on what defines each stage of maturity, and we are able to incorporate this as part of our consulting methodology.

It is interesting that the technology business is at last starting to move toward definitions that business people would recognise. Our Three Pillars definition – at least in relation to products – was summed up neatly by Paul Maritz, CEO of Pivotal – the Big Data division of IT storage giant EMC, who recently described Big Data as follows:

“Today, thanks to the technologies known as “big data” computers can capture things as they are happening and “can affect events as the events are unfolding.”

So, contrary to all the media hype, “Big Data” is really neither “Big” not about “Data” – it is about Questioning (in new ways), Accelerating (shorter time to business impact via insight and automation) and Transforming (using analytical insight to change culture – amplifying business intuition, not replacing it).

About the author: Mike Fish is a Director of BigData4Analytics Ltd, a Big Data Management Consulting firm – vendor-neutral and exclusively focused on helping enterprises make sense of Big Data.

Related Items:

Five Steps to Demystify Big Data Analytics

If You’re Missing Fast Data, Big Data Isn’t Working for You

Five Ways CDOs Turn Data into Dollars