DDN Weaves a Little HPC Magic Into Its First Commercial NAS Product
DataDirect Networks is a well-known provider of high-end storage solutions for HPC applications that rely on fast parallel file systems, like GPFS and Lustre. The HPC world will definitely benefit from today’s launch of the next-gen DDN14K line. But with the new GS14k, the company is hoping to make a big introductory splash into the network attached storage (NAS) pool by speeding the flow of big data across NFS and CIFS channels.
The new GS14k is one of several storage devices DDN is rolling out based on its next-generation DDN14K platform (previously codenamed “Wolfcreek”), which has been in development for the past three years. The DDN14K platform was designed to address the ever-growing data storage needs of business, governments, and research institutions, with a particular focus on HPC, hyper-scale data center, and big data analytics applications.
DDN’s first next-gen offering is the SFA14K “hyper-converged storage platform.” This new block storage device is designed to combine the latest hardware technologies–like SAS disks, NVMe solid state drives (SSDs), a PCIe-based internal fabric, and InfiniBand/OmniPath external interconnects—with DDN’s proprietary SSA operating system software that glues all that hardware together.
Support for OpenStack, Lustre, and GPFS, as well as storage density of more than 7PB per rack, will help the SFA14K attract its fair share of customers in DDN’s traditional HPC and hyperscale data center hunting grounds, as well as within Hadoop.
But this isn’t just an iterative improvement over its predecessor, the SFA12K, or its competitors. With more than 60GB per second of throughput and the capability to process more than 5 million input/output operations per second (IOPS) in a single 4U device, the SFA 14K will hold a 2x to 4x performance edge over competitors, says DDN’s CMO and EVP of Product Management Molly Rector.
“It’s a nice step up in performance,” Rector tells Datanami. “It gives customers the flexibility to not have to go buy an all-flash array for IOPS and go buy a scale-up solution for throughput. We put these capabilities in the same box to be able to accomplish both. That’s part of the magic of what our SFA operating system does: Deliver both throughput and IOPS in the same system.”
DDN’s second new storage offering is the GS14K, which is the company’s first foray into the scale-out NAS solution market. NAS storage systems are used frequently in business settings, and serve as the home for the CIFS- and NFS-based file systems serving data to business apps running on Windows and Linux.
While the NAS approach is incredibly popular and used throughout the computer industry, it’s not especially performant, especially compared to the parallel file systems that are commonly used in the HPC and technical computing markets. DDN decided to see how it could speed up data access in a NAS environment.
“We’ve done quite a bit to uplift those protocols [CIFS and NFS] and get the same kind of performance that you needed to go to a parallel file system client for before,” Rector says. “We’ve put quite a bit of new technology into it to make it easier to attach to in a non HPC-application environment, as well as added data management and security features that we need for real scale-out NAS solutions.”
While there isn’t much DDN can do about the actual CIFS and NFS protocols themselves, the company used a combination of SSDs, the PCIe fabric, and its internal software code to eek out performance boost. The company is touting 4GB per second throughput for each NFS or CIFS client via InfiniBand FDR. It’s a far cry from the 60GB per second of throughput the SFA12K can pump over Lustre/GPFS, but it’s much better than what was available previously.
This is how the DDN14k platform really shines, Rector says. “You can plug into a 40GbE controller but if the software that’s doing all the conversation only talks 2GBps, that’s the limiting factor,” she says. “Often that’s what the NFS or CIFS protocols have been doing. They’re heavy. They’re very slow. They’re not optimized for this kind of data set. They’re a bottleneck within a very high performance environment historical.
But increasingly, even customers in the commercial sector are demanding that extra performance, particularly when it comes to big data analytics. “Instead of storing data for long-term retention in case they need it, they’re actively using it all day long, so they need a lot more performance out of it,” Rector says.
In addition to the core DDN14K technology, the GS14K features enterprise features, such as support for backups, snapshots, encryption, and moving data to tape. These are not features that are typically needed in commercial or government HPC installations, but they are required in highly regulated private sector.
“It’s the first time we’ve actually positioned [the DDN platform] as a scale-out NAS,” Rector says. “We’ve always been reluctant to do that because we didn’t have some of the enterprise data protection features that are really necessary. So this is the first time we’ve entered the scale-out NAS space.”
The GS14K also features tiered storage, enabling hot data to be moved from the SSD cache to spinning disk as it cools off, and eventually archived to tape or the cloud as its requested less frequently.
DDN plans to announce additional members of the DDN14K product family next week at the Supercomputing show in Austin, Texas.