Industry Leaders Come Together And Endorse DAOS Foundation To Accelerate Nextgen HPC And AI/ML Workloads

(pathdoc/Shutterstock)

With the unprecedented growth of data, distributed storage systems have become a major bottleneck in data centers. Common challenges include difficulty managing large datasets, high latency data access, and lack of query capabilities. Distributed Asynchronous Object Storage (DAOS) has emerged as a solution to support high-performance computer (HPC) and AI applications. 

The Linux Foundation, a non-profit organization that supports Linux developments and open-source software products, announced the launch of the DAOS Foundation, which is aimed at advancing the governance and development of the DAOS project.

Distributed Asynchronous Object Storage (DAOS) is a high-performance open-source storage system that aims to redefine performance for a wide spectrum of HPC and AI/ML workloads. DAOS is designed to support high I/O operations per second (IOPS), low latency deployments, and high bandwidth. It offers next-generation data center workflows that combine AI, data analytics, and simulation. Unlike traditional storage stacks that were primarily designed for rotating media, DAOS is architectured to support new NVM technologies. 

DAOS harnesses the power of commodity hardware to optimize both cost and performance. It supports block, object, and file storage for a diverse range of application interfaces. In addition, it offers advanced features such as end-to-end data integrity, access control, and self-healing. 

The founding members of the Linux Foundation include Argonne National Laboratory (ANL), Enakta Labs, Google Cloud, Hewlett Packard Enterprise (HPE), and Intel. The DAOS Foundation has been endorsed by the founding members. 

(Den Rise/Shutterstock)

Intel Corporation will donate the DAOS source code to the DAOS Foundation and will remain a primary contributor to the future development of the project. “Intel is taking a pivotal step forward, solidifying our dedication to open-source technology,” said Intel’s Hillarie Prestopine. “The DAOS Foundation will drive innovation in storage software, delivering what is essential for AI and high-performance computing.”

“The DAOS storage system stack provides the core high-performance storage system for Argonne Leadership Computing Facilities latest supercomputer, Aurora,” said Kevin Harms, ALCF-4 Technical Director at Argonne National Laboratory. “DAOS has performance 20x greater than our current file systems and will provide better performance across a broad range of I/O workload allowing application science codes to achieve high performance with less effort.”

“Enakta Labs takes great pride in its role as a founding member of the DAOS Foundation,” said Denis Nuja, CEO at Enakta Labs. “We consider it a privilege to actively contribute and collaborate with the open-source community in advancing next-generation storage technology. DAOS serves as the foundation for the Enakta Platform, and we are excited to introduce this cutting-edge, high-performance storage system to the market.”

In August 2023, Google Cloud, one of its founding members, introduced new storage offerings, including Parallelstore, which is designed for demanding HPC and AI applications that use GPUs. Paralletstore is based on the DAOS architecture and offers a way to ensure that all GPUs have equal access to storage and that GPU resources are wasted while users wait for storage I/O. 

Commenting on the launch of the DAOS foundation, Sameet Agarwal, Storage VP/GM, Google Cloud said  “Google Cloud is delighted to be a founding member of the DAOS Foundation. This world record-setting technology underpins our new storage service, Parallelstore, and we are extremely excited to accelerate the growth of HPC and AI/ML workloads in the cloud.”

The DAOS Foundation offers General, Associate, and Premium membership levels, and it is open to anyone willing to contribute to the project. Each membership tier provides a different level of participation in technical working groups and governing boards. 

Related Items 

Why Object Storage Is the Answer to AI’s Biggest Challenge

IBM Unleashes the Potential of Data and AI with its Next-Gen IBM Storage Scale System 6000

Quantum Announces Next-Gen Cold Data Storage Solutions, Simplifying Cloud Integration