January 16, 2020

Container Interface Delivers Persistent Storage

As application containers took off in the mid-2010s, persistent storage emerged as a key barrier to adoption. With containers now entering the mainstream for handling machine and deep learning applications, more vendors are focusing on storage interfaces that can link huge data volumes along with flash memory to container orchestration platforms.

Among them is VAST Data, which this week announced general availability of a container storage interface designed to deploy and manage storage services across a range of standard container platforms. Along with making the shift to flash memory more affordable, the company said Thursday (Jan. 16) its storage interface enables “programmatic provisioning” of persistent storage volumes on container orchestration platforms like Kubernetes and Red Hat OpenShift.

(Separately this week, Red Hat released its latest version of OpenShift container storage, billed as an “enterprise Kubernetes platform” linking to data services offered by multiple public cloud vendors, including Amazon Simple Storage Service (S3). The accompanying object storage and scaling are touted as supporting data-intensive Kubernetes workloads running across hybrid cloud deployments, Red Hat said.)

VAST Data’s interface builds on the New York-based storage vendor’s “universal storage architecture” released in February 2019. The framework incorporates 3D Xpoint and quad-level cell flash memories along with NVMe-over-fabric.

The company said its storage algorithms are designed to reduce the cost of transferring data to those flash technologies, thereby enabling all-flash applications running in containers.

Jeff Denworth, VAST’s co-founder and vice president of products, said the storage platform implements storage CPU pooling and remote direct memory access to boost the performance of persistent storage volumes backed by the Network File System protocol. The result is a storage interface suited to leveraging containers for AI and other data-intensive workloads.

“We can now enable enterprises to bring all-flash performance and capacity at the price of [hard-disk drives] to container platforms that have adopted” VAST’s container storage interface, Denworth added.

The company further claims its container interface can stream data into single containers at a rate of nearly 9 Gb/sec., or four times the speed of legacy network-attached storage. Early customer testing of VAST’s storage interface driver and its underlying framework yielded a three-fold performance boost over all-flash storage-area networks.

The combination of persistent storage with all-flash technologies and storage fabrics also underscores how Kubernetes and various public cloud storage frameworks such as Amazon S3 are overtaking traditional approaches like Hadoop for applications like training AI models.

VAST Data’s storage interface driver for Kubernetes is available on Docker Hub.

Recent items:

AIOps and the Kubernetes Revolution

More Databases Running in Containers, Survey Finds

Cloud Looms Big for Big Data in 2020