Follow Datanami:
June 29, 2016

The NoSQL Tech Behind Vivint’s Smart Home


Here at MongoDB World in New York City, several providers of Internet of Things (IoT) solutions are sharing stories of how they use the NoSQL database. One of the most interesting IoT use cases involves Vivint, which is the largest provider of smart home solutions in the country.

Before pivoting to the burgeoning smart home business in 2010, Vivint was a home security company with about 700,000 customers. It had key partnerships with two firms, one that supplied the in-home hardware devices and another that ran the SQL Server database that stored data from the devices.

When Blackstone Group acquired the Salt Lake City, Utah company in 2012 for $2 billion, the company used a portion of the money to re-platform the technological underpinnings of its service to prepare for growth. That involved creating new in-home hardware devices, as well as a new software stack to enable customers to control aspects of their homes. The timeline for doing all this? A skimpy 12 months.

Vivint brought in an experienced technologist named Nicholas Brown to be its principal software architect and to build the software powering the new system. Brown had over a decade working with relational and NoSQL database technologies, but for this project, there was no question which database Vivint would use.

“We chose MongoDB for a couple of reasons,” Brown tells Datanami in an interview at MongoDB World this week. “First of all, we had very little time to do this. We had to get all the hardware technology and software developed within one year…So we had to build it all simultaneously.”

Getting hardware and software developed at the same time was a trick–especially considering the advanced lead time that hardware engineers and manufacturers demand–but modeling the data turned out to be a relatively simple exercise in MongoDB’s document database, Brown says.

“We took the simplest approach,” he says. “We took every system and its attached devices and its associated users and modeled that as one monolithic document. We took advantage of the structure and the flexible schema that Mongo allowed. Implementing that, creating our data access layer, and consuming that data for an application–it’s all very object oriented, just like the languages that are consuming it. So the implementation time is significantly reduced” compared to using relational technology.Vivint

Brown could have modeled the data using relational technology. But then he would have run into the second main advantage that MongoDB provides: horizontal scalability.

Oracle would have taken us a long way. You can certainly scale up with Oracle, but it would have been very expensive and ultimately it too would have been insufficient for the amount of load we were going to throw at the database,” Brown says. “It would have taken a lot longer and the performance characteristics are such that we would have had to implement some sort of application-level partitioning in order to get the horizontal scaling. Scaling up simply wouldn’t have done it.”

In his previous job at Adobe’s Omniture division, Brown was involved in a research and development project that exposed him to MongoDB. That knowledge enabled him to shortcut the evaluation process at Vivint. “I knew it would perform at scale,” he says. “We put it through the paces pretty quickly, in a matter of weeks.”

Fast forward four years, and today Vivint is providing home automation solutions to more than 600,000 customers, making it the single largest provider of general purpose smart home solutions in the U.S. (the company doesn’t compete directly with “hero products” like the Nest thermostat, which Google bought in 2013 for $3.2 billion).

Every time a Vivint customer turns on a light, opens a door, or turns up the air conditioning, that event is transmitted and recorded in the MongoDB database. Vivint also collects geolocation information from its smart phone application, in addition to collecting state information from the individual devices and individual home controllers.

All these events add up. All told, about 10 million Vivint devices are transmitting more than half a billion messages via RabbitMQ to its MongoDB database every day. The cluster, which spans 50 machines, encompasses 10 shards, and accounts for about 5TB of data across a trailing 45-day window–serves event data to an in-memory state processor Vivint developed internally.

Vivint Sky Control

The Vivint SkyControl panel

This MongoDB system–which Brown estimates requires about 5 percent of one worker to manage–is purely operational. For analytics, the company relies primarily on an Apache Kafka message broker to feed MongoDB event data into an Apache Hadoop cluster, where data scientists use using Apache Spark and machine learning algorithms to build the predictive models that put the “smarts” into the smart home system.

The consumer, of course, isn’t aware of all this big data magic behind the scenes. All they know is that Vivint seems to know whether anybody is home, what temperature they like the house, and when to unlock the front door.

The challenge for Vivint is to keep the cost down (subscriptions start at about $66 per month, and although installation is free, the cost of equipment runs into the thousands of dollars). Relying on off-the-shelf technology like MongoDB certainly helps in that regard.

“This is all mass market stuff,” Brown says. “There are home automation companies out there…where you can spend $50,000 or $100,000 to get the smart home of your dreams. We’re not that. Our systems have a monthly subscriptions that’s very low. But we curate our experience and that’s a key point to delivering these things.”

This is all just the beginning. The company has already integrated with Nest and allows voice control via Amazon‘s Echo, but Brown and his team at Vivint are putting the finishing touches on new offerings that will extend Vivint even further.

Eventually, the plan calls for Vivint to offer things like in-home audio that follows you from room to room, a GPS device that can be installed on a child’s backpacks to track their whereabouts, GPS devices to track cars location and speed, and ways to monitor the well-being older people who may need assistance from time to time. (Eventually, the company may even add household pets to the mix, which will allow us to finally crack their secrets.)

“It becomes smart life,” Brown says. “We’re really interested in delivering a lifestyle experience. It’s not just about cool flashy features in your house. We want really to make your life simpler.”

Related Items:

MongoDB Struts Its NoSQL Stuff in NYC

What 2015 Will Bring for Big Data

Home Automation Driving Data Innovation