October 11, 2013

Unstructured Video Data Creating IT Challenge as Volume Explodes

Isaac Lopez

A flood of high-definition video surveillance cameras that are being installed by enterprises around the world is creating a big data challenge that looks set to explode over the next four years, says analyst firm IHS Inc. in a new report. The results, according to the firm, is the increasing adoption of big data technologies, as well as innovation in how such data is handled and stored.

According to IHS, the world is currently producing 413 Petabytes (PB) per day by all of the HD video surveillance cameras that have been installed in recent years – enough to fill 92.1 million single-sided, single-layer DVDs. In a new report, titled “Enterprise and IP Storage used for Video Surveillance,” the research firm projects that by 2017, that number will more than double to 859 PB.

“HD-compliant products are set to account for an increasing share of video surveillance camera shipments during the next four years,” said Sam Grinter, senior surveillance analyst at IHS. “These cameras are gaining acceptance because the quality of their video can be superior to standard-resolution products that formerly dominated the market. But because each HD camera produces far more data than each standard-definition camera, the quantity of data generated by the surveillance market is growing to massive proportions.”

The massive amounts of data generated by these HD cameras is creating an IT problem for the businesses installing them because these IP-enabled cameras are churning out data that needs to be routed through the network, creating volume and velocity problems for network administrators who manage this traffic.

Aside from bandwidth issues, eventually this data needs to be landed somewhere in the datacenter. In many cases these businesses want to store the HD video data for extended periods of time – a proposition that can prove to be expensive, as the single largest expense for these video systems is often the storage.

Storage aside, another issue is what to do with all of this data once it has been landed. In an article earlier this year, Dom Nessi, deputy director and CIO of Los Angeles World Airports (LAWA), said that LAWA’s system contains over 2,000 cameras and 33 PB of disk. That data is stored for 13 months, because no one can predict what data might be important down the line.

According to the article, LAWA is using video surveillance software from Israel-based NICE Systems to help manage the video data. However the organization is still exploring ways to take advantage of analytics – something that proves to be a real challenge when you consider the complexity of unstructured video data and the need for the algorithms to discern meaning from what the video is showing.

An technology field is emerging around this called Video Content Analytics (VCA), which helps by enabling cameras with an algorithmic visual cortex, giving the camera “brains” to know when and what it should be recording. While standard motion detection is a big part of VCA, other elements are involved, such as object detection, and face and number (i.e. license plates) recognition.

 There are also several initiatives underway to address the glut of HD videos and to cut video data down to more manageable portions. The data compression algorithm known as the High Efficiency Video Coding (HEVC) standard, (or more commonly, “H.265”) is said to double the data compression ratio of its predecessor H.264 standard.

The rise of this unstructured video data is just another example of the challenges that IT faces as the world moves further into the data age. We’ll look forward to following the advances in this space as new technologies rise to take on these challenges.

Related items:

IBM Hits the Throttle on Big Data Discovery With New Lab 

The Evolution of Data Management 

SQLstream Analyzes Data On the Flow

Share This