The Secret to Generating Value from IoT Data
Over the last 10 years, enterprises have been implementing IoT applications by connecting devices and “things” on their internal network (Intranet) to provide better customer service, reduce costs and more. By applying advanced analytics technologies designed specifically to handle the requirements of real-time big data analytics, many businesses today are converting the vast amounts of IoT data generated into actionable insights.
With more and more devices being connected to the Internet, the opportunity for additional value is growing rapidly as the projected number devices connected broadly across all of IoT continues to grow at a rapid pace.
These billions of connections and sensors collecting data from real-time operational events are creating booming growth in both the amount of data being created as well as the velocity at which the data is being collected. Analyst firm IDC has estimated that by 2020, the world’s data will grow to 44.4 zettabytes (one zettabyte = 1 billion terabytes)…and IoT is one of the biggest contributors to that growth!
To truly take advantage of the massive amounts of real-time data in the emerging IoT world, we need to transform our thinking, processes, and infrastructure to act in seconds – not hours and days.
Many companies have made progress by placing sensors throughout their factories, communication towers, supply chains, vehicles, etc. to capture real-time events. However, the value of this real-time IoT sensor data cannot reach its full potential if it takes days to actually derive actionable insights. What good is it just “knowing” that our manufacturing assembly line is experiencing product quality degradation? The real value lies in being able to identify the root cause of the issue quickly enough to avoid the high cost of re-work and product waste. Unfortunately, numerous business leaders and CIOs have stated that their legacy big data or Hadoop “general purpose” infrastructure is not equipped to handle the needs of their increasingly time-sensitive “real-time” operations.
From a database perspective, how can enterprises ensure success in real-time analytics for IoT? The first step is to think more holistically about the critical attributes of a database. To do so, we must examine the six key factors required to truly enable real-time analytics:
- Scalability to handle massive amounts of data, especially critical for IoT use- cases, where data will to continue to grow exponentially. Data always seems to grow bigger and faster than we initially anticipate.
- Velocity of data ingestion: IoT data is coming at us faster than most other sources of data, with more than 1 million rows per second. Databases need to be able to load data continuously without affecting query performance.
- Speed in performing queries for immediate insights. In addition to running pre-defined queries quickly, the ability to execute instant ad hoc queries is critical for many IoT use-cases (e.g. interactive root cause analysis).
- Flexibility and portability of the analytics platform: The ability to run anywhere – in the cloud, on-premise, embedded, etc.
- Distributed database: Increasingly, IoT use-cases call for the ability to perform geo-distributed queries (querying at the “source” of data) to avoid the transport of large data sets across expensive or poor network connections.
- Efficiency of the footprint: Does the database have “light” hardware requirements? Can it run on thin clients or even portable hardware? For example, the need for a 100-server cluster will inhibit many IoT use-cases and greatly increase the Total Cost of Ownership.
Bottom line? Choose the right tool for the job. Clearly, the requirements of today’s IoT applications are very different from the attributes of “general purpose” databases developed 25+ years ago. Choosing a database designed for the specific needs of IoT will ensure success today and in the future.
IoT is all about creating connections to capture real-time events, which enable companies to transform their “sense and respond” capabilities. However, without the right real-time analytics infrastructure, organizations will be not be able to move from “knowing” to “doing” fast enough to take advantage of the business benefits.
Real-time analytics is a foundational pillar for IoT, and the key to unleashing the value of IoT.
About the author: Peter Jensen is CEO of ParStream, a Silicon Valley-based IoT analytics company. Prior to ParStream, Peter was CEO of StopTheHacker, a security SaaS company focused on malware detection. Previously, he was VP of worldwide sales for Pancetera (acquired by Quantum) and Thinstall (acquired by VMware). He has also held senior sales and management positions with Oracle, Symantec, and VMware.