Follow Datanami:
August 21, 2015

Big Data Workloads in the Cloud: One Size Does Not Fit All

Ashim Bose

There is a bewildering array of big data platforms in the cloud today including multiple flavors of Hadoop and varied technologies from major vendors. The difficulty of selecting the “right cloud platform” is exacerbated by the fact that there are no established guidelines about how to configure the right cloud infrastructure to support a specific use case around a set of technologies and related workloads.

There is also a gap in the service levels needed by enterprise clients compared to those provided by most cloud vendors. Clearly, a “one size fits all” approach does not work. Taking that approach leads to unanticipated costs, risks, and delays in deploying and provisioning under or over used hardware and software and related services.

For example, a use case looking at six months of sales data to evaluate sales force effectiveness requires specific analytic software for structured data, compute, and storage aligned with appropriate performance needs. Also, the required service level agreements (SLAs) depend on the business criticality of the analytics. If it is more for “discovery” purposes, the SLAs could be relaxed to “two or three 9s” with an 8×5 support model.

However, if the use case is extended to offer real-time insight on demand to any sales representative globally on the client’s “propensity to buy,” the complexity of the use case increases significantly. The analytic software may need to cover both structured and unstructured data to analyze comments, social media and more. The compute and storage needs would need to be aligned with real-time processing of large data sets of varied types. The SLAs to support just got more stringent to be “five 9s” with a 24×7 support model as the analytics would need to be deployed in “production” and available on-demand globally in real-time.

Enterprises today have a variety of needs for both “discovery” and “production” environments in terms of software, cloud infrastructure and SLAs and consequently, what works for one organization may not work for another when it comes to platform selection. An enterprise needs to understand and align its specific needs for software, compute, storage, and SLAs to the use case and come up with guidelines to make this a reusable, easy to use process. This is convenient to do in the cloud-based model where provisioning can occur on demand, and has the following benefits:

    • Optimizing the use of software, compute and storage based on the use case
    • Paying for only what you use for both hardware and software
    • Having a full spectrum of infrastructure and associated managed services to adapt to various needs
    • Having a seamless migration path as needs change
HP_1

Guidelines to leverage pre-configured components of a cloud solution based on requirements can significantly reduce the time to provision a cloud solution for analytics

Recognizing that use case needs can change with time necessitating a need for a different solution, having a seamless migration path from one solution to another and/or integration across them is also key.  The change could occur on multiple dimensions including software, compute, storage and SLAs.  The cloud model selected needs to be able to accommodate these changes rapidly, facilitated by migration process and/or integration scripts and SLAs for the migration itself.  In addition to the basic considerations in the above figure, it is assumed that other key cloud features such as security, connectivity, backup, disaster recovery and self-service are also provided.

Enterprises have a wide spectrum of needs for big data analytics solutions in the cloud.  One size does not fit all, but having pre-configured building blocks to develop a solution for a specific need can significantly reduce the time, risk, and cost to deploy a solution in the cloud.

Selecting a cloud vendor with the flexibility and agility to rapidly respond to the changing needs is a key foundational element.  Enterprises need to evaluate cloud providers for their ability to flex rapidly to accommodate their needs, and provide capabilities supporting operational integration and migration within enterprise frameworks. They also need to develop the building blocks and blueprints to be able to rapidly deploy cloud solutions and have the ability to migrate and integrate across them.

Ashim Bose

About the author:  Ashim Bose is Senior Director, Technology & Platforms, HP Enterprise Services Analytics and Data Management.

 

Related Items:

Tracking the Rapid Rise in Cloud Data

Five Reasons Machine Learning Is Moving to the Cloud

Datanami