Should Your Analytics Be ‘Real-Time’?

It’s widely acknowledged that the quicker an organization can collect, analyze and act on data the more of an advantage they will have in the marketplace. In fact, many organizations believe that the data they’re collecting must even be “real-time”—which is to say accurate up to the very second. But, is the added benefit of performing analytics on real-time data worth the effort? Here’s how to determine if real-time analytics should be part of your business intelligence strategy.

Let’s first make sure we’re speaking the same language. I’ve seen real-time defined in a wide variety of ways—and in most cases, the definition offered is the one most self-serving. I find that people add the word “real-time” to batch processes to create a sense of urgency, but that never works, and the technologies they require are different. Therefore, we are better off sticking to the distinct classifications of batch, expedited batch, and real-time. For the sake of this argument, we’ll focus on the latter.

Do you need to know now, or know what’s happening now?

When evaluating the possibility of real-time analytics, the first question you should ask yourself is: What is the most important “now”?

Sometimes you need information and facts to make decisions now, but those facts needn’t be occurring right now. This does not necessitate real-time analytics; it’s simply needing support at the point of decision-making—important, but it does not require data that’s streaming in real-time.

But when you need data on what is happening “at the very second” (like when Uber pulls data from its drivers to locate the car that is nearest you), data that’s any older isn’t nearly as useful. In this case, that’s when you really do need real-time analytics.

Another great example is in shipping and logistics, where sensor data is being used to optimize shipments in real-time. It’s an industry in which time and space are literally money. Trucks are being loaded from a warehouse with a variety of packages, and sensors track the loading of individual packages into each truck as it’s happening. Any free capacity is updated in real time, allowing managers to maximize cargo and improve the bottom line.

Even if I don’t need to know what’s happening now, it would be better to go real-time anyway—right?

Not necessarily. Enabling real-time analytics is an intensive process, and if you don’t need to know what’s happening each second, it’ll be a significant waste of resources. For example, does a retail store manager need a real-time dashboard of every transaction? Would it help them make better decisions? Most likely, no. Retail is a volume business, so the addition of one transaction is inconsequential.

That said, thanks to the growth in Internet of Things (IoT) technologies, real-time analytics will begin presenting additional opportunities for organizations around automation initiatives. IoT will automate a large part of human decisions; think of it like what the frontal lobe does in the human brain.

“System 1” thinking is automatic for fight-or-flight decisions. It has helped humans through evolution. “System 2” is the deliberate, slow system, more informed, but often leads to analysis paralysis.

IoT is making sure that the automation is taking place (a la “System 1”). Real-time analytics will allow for a constantly evolving decision automation process in the same way as it happens in System 2 thinking in the human brain: by understanding the context (data) around the decision and how changes in the system affect future decisions.

The field of law enforcement provides a perfect example of how real-time analytics can enable better informed “System 1” -type decisions.

When officers are directed to a crime scene, it is vital that they have accurate real-time information on what to expect, such as whether there are weapons or other dangerous factors to consider at the scene. This information, which helps to save lives, is often probabilistic data derived from a very robust analytic process designed to deliver real-time information. Operatives in the field do not have time to analyze the vast amount of data available (a la “System 2”). Instead they have to assess the situation based on real-time analytics delivered to them at the point of decision-making.

So, is it the right time for real-time?

If your decision-making process could be aided by having up-to-the-moment data in reach, go for it. Otherwise, while you may not need real-time analytics today, consider decisions you could automate in the near future—the combination of IoT and real-time data technologies promises to be a game-changer for any organization that can capitalize on it.

About the author: Dr. Rado Kotorov is the Chief Innovation Officer for business intelligence (BI) and analytics provider Information Builders and co-author of Organizational Intelligence: How Smart Companies Use Information to Become More Competitive and Profitable. He is responsible for analyzing market and technology trends, aiding in the development of innovative product roadmaps, and creating rich programs to drive adoption of BI, analytics, data integrity, and integration technologies. Rado strives to make BI and business analytics more accessible, intuitive, and collaborative through the adoption of innovative Web 2.0, advanced visualization, predictive modeling, search, and mobile technologies. He has a PhD in decision and game theory, and institutional economics from Bowling Green State University. 

Related Items:

Yelp Open Sources Data Pipeline That Saved It $10M

The Real-Time Future of Data According to Jay Kreps

What’s Driving the Rise of Real-Time Analytics