Follow Datanami:
February 16, 2012

Stacking Rocks for Big Data

Robert Gelber

Prior to big data’s grand entrance, using single servers with a RAID array and possibly tape backups was an accepted method for storing, accessing and hopefully analyzing relevant data. That environment has changed, as more demand for horsepower in the form of HPC clusters, dynamic file systems and powerful visual analytics tools are on the rise.

Cluster management vendor StackIQ saw the increase in demand for more integration of big data analytics with HPC horsepower and released Rocks+ 6.0 back in November. The goal of the application was to provide a soup to nuts solution that incorporated all necessary cluster components under one roof in what the company described as “big infrastructure”.

The software provided support for cluster admins to install a variety Hadoop distros Cloudera, MapR, Hortonworks and would automatically provision and manage Hadoop components including MapReduce, Hive, HBase, HDFS and Zookeeper. Included, was a real-time visualization tool that allowed managers of a cluster to see the progress of the parallel install.

The software also featured a plug-in system that allowed for customization of “rolls” or modules and touted features such as support for multiple instances of MapReduce to run on the same file system. StackIQ mentions that the software’s architecture allows for development of various partner modules.  Some of these modules, or rolls included the Rocks+ Hadoop Roll, Rocks+ Workload Management Rolls and the Rocks+ High Performance Computing and Analytics Rolls. Oddly enough, there doesn’t seem to be any mention of a corny rocking and roll joke anywhere on their site. 

The UC San Diego neighbor has just updated Rocks+ to version 6.0.1. So what changes can current 6.0 users expect? In their release, StackIQ touts graphical management features for Hadoop and support for an extended group of HPC workload managers.

Updates to their rolls, including CUDA, Solarflare, Moab, OFED and Univa Grid Engine are also part of the update. Certainly, grid engine users will be happy to receive hear this news.  Also, a new pricing scheme to allow for more custom packaging of Rolls based on the needs of cluster managers.

Ultimately, Stack IQ’s goal appears to make cluster management as inclusive to technology and people as possible. The two most recent Rocks+ software revisions have focused on adding support for big data distributions and HPC workload managers.

Tim McIntire, the president of the StackIQ has even gone through the task of creating a YouTube channel providing video walkthroughs of how to setup physical and virtual clusters using Rocks+ software, and for a the bonus, he even has a video tutorial of how to setup a Hadoop cluster in Amazon’s EC2.

Related Stories

StackIQ Update Rocks Big Data

Datanami