Follow Datanami:
September 30, 2013

IDC’s View on Cray Cluster Supercomputers for HPC and Big Data Markets

Written by Maria McLaughlin

It’s well known that the high performance computing (HPC) market has grown quickly over the last couple of decades. In fact, the HPC market has tripled in size from $3.7 billion in the mid-1990s to $11.1 billion in 2012 according to IDC.

But what may surprise some industry observers is the furious growth of cluster supercomputers. In 2003, these systems accounted for only 20 percent of all HPC system revenue; but from 2007 onward, clustered systems have grown to represent roughly 65 percent of total HPC system revenue, according to IDC. In addition, we’re likely to see the demand for cluster supercomputers continue to climb over the coming years as Big Data increases the demand for systems that can handle data-intensive workloads.

A recent IDC white paper , sponsored by Cray, (“Cray Cluster Supercomputers Take Aim at Big Computing and Big Data Challenges”) discusses the factors driving the growth of the global market for large-scale technical computing clusters. The paper also evaluates Cray’s cluster supercomputer solutions in relation to the needs of this market and assesses the challenges and opportunities Cray faces.

Clusters come in many sizes. According to the IDC White Paper, clusters costing half a million dollars and up represent about two-thirds (64–65 percent) of HPC servers sold today. Furthermore, the IDC White Paper predicts that in 2016, the market for large-scale clusters in the supercomputer and divisional segments will reach $7.3 billion and the server market for high-performance data analysis will amount to about $1.3 billion.  Clusters account for a large majority of system sales in the lower half of the HPC market (systems under $250,000), and large-scale clusters have a substantial footprint in the supercomputer and divisional segments (systems over $250,000), where they share the limelight with more tightly coupled HPC systems.

One of the major factors driving the growth of cluster supercomputers has been the rapidly improving performance and efficiency of these systems. Radically increasing performance from industry-standard interconnect technologies (like InfiniBand and Ethernet) plus much-improved software parallelism mean these systems can scale high enough to handle the massive compute problems that used to be the exclusive province of traditional supercomputers. These improved technologies make them ideal for data-intensive computational workloads.

The design of cluster supercomputers also makes them attractive to customers looking for a highly flexible solution based on industry-standard building block platforms with highly customized components — particularly when it comes to small or mid-sized configurations.

The choice between a cluster supercomputer and a traditional supercomputer comes down to customer needs. Some problems can be addressed with cluster systems while others need the more customized solution provided by a tightly coupled and highly integrated supercomputer.

At Cray we strive to find the right computational tools to meet the customer need. While most people know Cray as the leading provider of tightly coupled and highly integrated supercomputers, our acquisition last year of the highly recognized cluster supercomputer vendor Appro has expanded our offering to include cluster supercomputer solutions and expertise. With the addition of cluster systems (now known as Cray CS300 product line), Cray offers a full slate of supercomputer solutions to serve the wide ranging needs of customers in HPC, data-intensive computing and big data. This also positions Cray to provide solutions for HPC customers in the divisional segment.

The Cray CS300™ Cluster Supercomputer Product Line

We also have a lot of momentum in the cluster supercomputer space. Our newest CS300 systems offer a wide range of configurations to fit unique customer requirements with highly efficient air-cooled and liquid-cooled cluster solutions. The CS300 cluster supercomputer series can be configured with a wide variety of interconnects, including various torus schemes and utilizing QDR/FDR InfiniBand. These systems are based on Intel® Xeon® E5 processors, with support for compute accelerators like GPUs and the newest Intel® Xeon® Phi™ co-processors.

One of our latest cluster innovations is the CS300 liquid-cooled system that can help reduce datacenter PUE (power usage effectiveness) ratings to a level of 1.1 or lower — a vast improvement relative to what most datacenters are seeing today. The CS300-LC model captures and removes up to 80 percent of the heat generated by processors and memory via directly attached heat exchangers. This is a warm water-based solution, meaning that customers don’t need to use expensive chillers to pre-cool the water in the liquid loop.

To meet the demand for high performance big data applications, we’ve also introduced Cray cluster supercomputers for Hadoop®. Built on optimized configurations of the Cray CS300 cluster supercomputer series, these systems feature Linux™ OS, workload management software, Advanced Cluster Engine™ management software and the Intel® Distribution for Apache Hadoop® software  —  all integrated, optimized, validated and supported by Cray.

The Cray CS300 line of products does not stop there – we also offer preconfigured large memory and shared memory systems that are ready-to-go solutions targeted to specific markets.

Portions of this In the Spotlight advertorial have been adapted from a blog written by Maria McLaughlin.

Datanami