Follow Datanami:
September 1, 2016

Defining Edge Analytics from a Data Decentralization Perspective

Simon Moss

Revolutions in processing power and analytical modelling, distributed through massively democratized technology, have opened up countless new possibilities. But these revolutions in technology design have not been matched by our approach to big data. For the last 40 years we have accepted a common prerequisite for the creation of value from our data, and that’s the idea that it has to be homogenized and centralized before we can do anything useful with it.

The logic of this supposition is seductive, but the reality is that it doesn’t work.

Through 2017, as many as 60% of big data projects will fail to go beyond piloting and experimentation and will be abandoned, according to Gartner. Advanced analytics is largely a matter of “changing mindsets and culture as it is about acquiring tools and skills,” they say.  (Interesting to note how the research firm defines ‘advanced analytics’ as things that include machine learning, pattern matching, visualization, simulation, complex event processing, and neural networks.)

But the failure of big data projects does not stem from a problem with the analytics; it’s the result of a fundamentally flawed centralization approach that’s so expensive and difficult to achieve that it can’t possibly deliver a high enough level of ROI.

The Risks of Centralization

Large enterprises, financial institutions, pharmaceutical companies, and government departments have systems that go back decades. They may have data in 50,000 different locations and in 10,000 different representations. It’s incredibly expensive and time-consuming to try and normalize all this data and bring it together. It requires a huge investment in infrastructure and the probability of success is low.

Every time you move or duplicate stored data, you are increasing the surface area for a potential attack. It’s a major and unnecessary security risk. By storing and transferring data multiple times you’re also exponentially increasing staff, storage, and compute costs. The time that it takes leads to degradation in the quality and relevancy of your data, so the business value declines.

Consider that most companies are growing more diverse and distributed all the time and you can see that centralization is not the answer. There has to be another way.

Living On the Edge

The mobility trend and the rise of IoT has led to massive data flows being generated at the edge. Huge volumes of raw and intermediate data are spilling into increasingly distributed application environments that encompass legacy infrastructures, cloud-based resources, and mobile computing platforms. The proliferation of data sources and end points continues to push centralization beyond reach, transforming the dream of homogenized data into a costly nightmare.IOT_graphic

What if we turn the whole thing on its head? Why not take the analysis to the data? We can compress the timeline and shorten the distance between the components of value – the data, and the creation of value, to arrive at results. We can orchestrate intelligence on the edge, dipping directly into the data topography of applications, processes, and operating models through a layer of analytics fabric that sits on top.

There are many prerequisites for this new approach to succeed. Compatibility is key, because it has to cope with current tools, legacy systems, and new technologies. It should be lightweight, scalable, and easy to deploy across different environments. Security must be a cornerstone, with unnecessary data movement minimized and a clear audit trail. An intelligent system will adapt and ensure that a high level of performance, security, and integrity is maintained based on available resources.

We Are Creating the Big Data Problem

By re-examining the supposed big data problem, we may realize that it was actually a distribution and diversity problem all along. We’re moving all these different data sources into one single location and then wrestling to find a way to deal with the giant volume we just created. We’re disrupting all these interactions and the minute we normalize and centralize the data the value is degrading. Why not design and deploy our analytics at the edge where everything is happening?

We understand the problem we’re trying to solve, all that’s really holding us back is that the components of the solution are widely distributed and incredibly diverse. The cutting edge analytics and processing power that offers so much potential is completely hamstrung by a deployment and centralization model that’s 40 years old. Mobile analytics align with the proliferation of data across an expanding ecosystem. Focusing on the edge can deliver the speed and agility that’s required to innovate.Simon Ross

About the Author: Simon Moss is Chief Executive Officer for Pneuron Corporation, a business orchestration software provider. He was previously CEO of Avistar and CEO at Mantas, later acquired by Oracle. He served as Partner at Price Waterhouse Coopers, and was co-Founder of the Risk Management Services Practice at IBM. Moss is also on the Board of Directors for C6 Intelligence. Contact him at [email protected].

Datanami