Couchbase Eyes IoT with Mobile NoSQL Database
Couchbase sees the new mobile NoSQL database it rolled out yesterday giving developers powerful new tools for building applications that store and process large amounts of data on smartphones, tablets, and other devices that are part of the Internet of Things (IoT).
Couchbase Lite is a full-featured NoSQL database that stores data in the JSON format and runs natively on iOS, Android, and Java; a .NET version for Windows devices is in beta. The mobile database connects to a primary Couchbase database via the Sync Gateway, a new component that runs next to the primary database on the cloud-based server. Together, the three products make up the Couchbase Mobile offering.
Developers use the same tools, APIs, and techniques for building mobile NoSQL databases as they do for developing server-based NoSQL databases, says Rahim Yaseen, senior vice president of product development at Couchbase. In fact, Couchbase Lite can be thought of as just another node in a NoSQL database cluster, since it uses the same data technology and protocols to distribute data as the full database.
The one exception to that characterization is the sync–the all-important data sync. When local data storage was required, developers would typically have to build their own data sync programs on top of SQLite, an embedded relational database that ships with all iOS and Android devices. Couchbase Lite brings its own pre-built sync, eliminating the need for developers to build their own. That will be a boon for developer productivity, Couchbase predicts.
But it wasn’t easy. In fact, finding the optimal way to keep the mobile and server databases synchronized was the hardest part of the whole project, Yaseen says. “Sync is a hard problem. It’s extremely difficult to build a sync,” he says. “We’ve spent three years building a very powerful general purpose sync mechanism.”
Besides “solving” sync, Couchbase sees several other forces aligning to drive developers to rely more on local storage and rely less on the cloud. “We started this project based on one prediction, which is that devices in the future are going to have enormous amounts of compute and memory in them. They’re going to be quad core, eight core, with half a terabyte of flash memory on them,” he says. “If you have that much compute and you have that much memory, your bottleneck, which is the network, is going to get worse.”
Couchbase wondered what would happen to network I/O if mobile devices had the capacity that laptops had just a few years ago. “If that’s the case, people are going to run local apps a lot more on the device,” Yaseen says. “Having chatty end-user apps just hitting the network all the time just to get every piece of data–we though that would be a bad, bad idea.”
Couchbase Lite actively manages the network connection for the applications and users it supports, and only updates data using an “append” database command, ensuring that only the bare minimum of data crosses the network.
The product supports three types of syncs, and enables developers to build applications that adapt to varying levels of network availability. The streaming sync is useful when the network connection is strong.
“That’s going to give you really low latency for applications that want that,” Couchbase architect Wayne Carter says. “We also have polling mechanism where you can set it up on a time-based sync. We also have push mechanism where you can let the device know they have a change and then they can go back up and get the change from the server.”
While the “mobile-first” phenomenon relied on REST services to move data across the network, the expected jumps in storage capacity will require a “local-first data strategy, Carter says. “This will enable applications that are much better, must faster, and much more reliable,” Carter tells Datanami.
The technology will be useful in consumer-facing mobile applications as well as the industrial IoT and healthcare. Couchbase is currently working with a large European carmaker that’s looking at using Couchbase Lite to power a preventive maintenance application.
“They have a lot of sensors in a car today and they want to be able to capture that data. But a car doesn’t have a network all the time,” Yaseen says. “They want to collect the diagnostic data, and essentially have people sign up for maintenance program…so when check engine light comes on, your service guy knows what that is before you even know.”
Another customer is looking at using Couchbase Lite to collect sensor data on oil rigs, which are also outside the range of cell networks most of the time. The oil rig company has been collecting sensor data for a long time, but storage capacity, bandwidth issues, and the all-important data sync complicated things.
“The oil rig company built their own flat file for this. And many people have been doing this with some flat file or some homegrown database, and have to spend a lot of time doing the sync,” Yaseen says. “We believe that type of application is mature enough for a general purpose product to be used today. Those [homegrown] systems are reaching the limits or their boundary in terms of not being able to handle a terabyte of data.”
As the big data phenomenon powers forward, mobile devices and the IoT will be called on not only to present information for consumption, but to capture data for use by thee server as well. Couchbase Lite will help ensure that databases are part of that. “This will spawn a new wave of mobile innovation for new applications that just couldn’t exist yesterday or weren’t as useful yesterday,” Carter says. “Impossible applications that you could not build yesterday are now possible.”
One example of this is peer-to-peer connectivity. Microsoft Open Technologies is using Couchbase Lite to power peer-to-peer communication in its Thali project. “We have been using Couchbase Lite in Thali, an experiment to determine through open source prototypes what it takes to make the web truly peer-to-peer,” says Jon Udell, Microsoft Open Technologies.