Aerospike Simplifies Global Data Management with NoSQL Update
Many large, global companies struggle to maintain centralized control of customer data while simultaneously adhering to country-specific data regulations, such as GDPR and CCPA. Now Aerospike is addressing that challenge with a new feature in its NoSQL database that gives customers more control over which pieces of data are replicated to localized database clusters.
Aerospike added an enhanced predicate filtering function, which it calls expressions, with the version 5.2 launch of its eponymous database last month. Expressions provides an advanced querying capability that essentially works as a filter to automatically reduce the amount of data that’s returned from the database. It’s similar to what some relational databases already offer, and Aerospike now is adopting the idea for its distributed NoSQL database.
With today’s launch of version 5.3, Aerospike is adding support for expressions to the Cross-Datacenter Replication (XDR) version of the database, which allows companies to replicate data among multiple geographically distributed databases in an asynchronous manner (as opposed to the multi-side clustering version of its database, which it launched earlier this year and which maintains a single, globally distributed database using synchronous replication among the nodes).
According to Srini Srinivasan, chief product officer and founder of Aerospike, support for expressions in XDR will allow large organizations to more efficiently move record-level data among the various Aerospike clusters that are involved in an XDR setup. That, in turn, will make it easier for those organizations to adhere to local data regulations without overburdening administrators.
“The most important thing is the fine-grained nature of this, because expressions can be applied at the record level,” Srinivasam says. “If the consumer says ‘Do certain things with my data,’ then you have to obey it, which means you have to capture it and then you have to filter based the settings of each record potentially.”
Expressions support in XDR makes it easier for organizations to automate the handling of data for large numbers of customers who are represented in the database. For example, if data regulations say that data associated with UK citizens should not be moved outside of that country, then an expression can be written to enforce that rule at the record level.
Before Aerospike supported record-level expressions, people would have to statically define the rules using tables, Srinivasam says. As people moved and as the rules changed, companies would have to update those tables, which usually involved replicating large amounts of data, and the creation of additional data silos. This raises costs and complexity for enterprises with operations in multiple states and countries, and leads to a less efficient business.
“I would say we are aligning to the more dynamic nature of what happens with data these days,” Srinivasam tells Datanami. “We now have features which enable you to easily manage your data at scale. All of this only matters at scale when have hundreds of millions of consumers and you have to comply with all these rules. It becomes hard if you do static mapping.”
By giving customers greater control over data rules, Aersopike customers can provide greater consistency to global database operations, while still meeting localized rules preventing some pieces of sensitive data from being moved. This will also benefit customers by minimizing the amount of data that gets replicated from site to site, thereby reducing network spending.
“The biggest challenge…global enterprises have, when they deal with data from people all over the world, is they don’t have a unified view, which means they can’t run an efficient business,” Srinivasam says.
Expressions gives customers “centralized control in terms of defining the same rules everywhere,” he continues. “It is not centralized control in terms of one database controlling everything, because by definition, this system is globally distributed. You have different clusters with different subsets of data, depending on which geographic area the cluster is deployed. Some data is shared. Even for the same user, a portion of the record can be shared globally, but some parts can’t be. So all of that can be enforced with these rules.”
Aerospike 5.3 introduces one other major new feature: an elimination of the 100-millisecond latency limit in multi-site replication.
When implemented as a single globally distributed database, Aerospike uses synchronous replication techniques to keep all of the nodes in synch (as mentioned above). This is particularly important in the financial services industry, when there is a high price to pay for losing even one transaction.
However, synchronous replication is more sensitive to network latencies that can’t be avoided due to physical conditions, such as the speed of local networks (not to mention physical laws, like the speed of light). By relaxing the latency requirements of its synchronous replication in multi-site replication, Aerospike customers can effectively put database clusters anywhere on the earth, and still keep them within a single, functioning database.
The feature was added due to one Aerospike customer that encountered difficulty in maintaining database consistency with nodes spread out between Europe and Singapore. “For some reason, it was going past the 100 millisecond limit,” Srinivasam says. “The cluster couldn’t hold itself together, so we just had to extend it to deal with pretty much any two points on the globe.”
For more information on Aerospike version 5.3, check out this blog post posted today to Aersopike’s website.