7 Things You Need to Do About Big Data in 2013
by Brett Sheppard, Tableau Software
What a year 2012 was for big data! From the White House to your house, it’s hard to find an organization or consumer who has less data today than a year ago. Database options proliferate, and business intelligence evolves to a new era of organization-wide analytics. And everything’s mobile.
Organizations that successfully adapt their data architecture and processes to address the three attributes of big data — volume, variety and velocity — are improving operational efficiency, growing revenues and empowering new business models. With all the attention organizations are placing on innovating around data, the rate of change will only increase. So what should you expect to do to succeed with big data in 2013?
According to Forrester Research: “Enterprises find advanced data visualization platforms to be essential tools that enable them to monitor business, find patterns, and take action to avoid threats and snatch opportunities.” The Forrester Wave™: Advanced Data Visualization Platforms, Q3 2012, July 17, 2012.
Visual analysis means exploring data visually. A story unfolds as you navigate from one visual summary into another. According to Dana Zuber, Vice President of Strategic Planning at Wells Fargo: “A well-crafted, thoughtful visualization makes the light bulb go off. You just don’t get that with a spreadsheet.”
A visual analysis experience has certain characteristics. It allows you to do two things at any moment:
• Instantly change what data you are looking at – this is important because different questions require different data.
• Instantly change the way you are looking at it – this is important because each view may answer different questions.
This combination creates the exploratory experience required for anyone to answer questions quickly.
Source: Tableau Software
The cycle of visual analysis is a process of getting data, structuring it one way, noticing results and asking follow-on questions. These follow-on questions might lead to a need to drill down, drill up, filter, bring in new data, or create another view of your data. And you’ll want to share and act on the data.
Without interactivity, you and your colleagues are left with unanswered questions. With interactivity, the data visualization becomes a natural extension of your thought process. Furthermore, interactivity allows colleagues to ask and answer their own questions, shortening the time it takes to convince colleagues and moving faster from insights to business results.
Source: Tableau Software. See and interact with the live web dashboard on your desktop, iPad or Android tablet.
Finding it hard to keep track of all of the new database vendors and open-source projects? Well, 2013 will even be more crowded.
Source: Matthew Aslett, The 451 Group, Updated database landscape graphic, November 2, 2012
Take a strategic approach by extending your relational and online transaction processing (OLTP) systems to one or more of the new on-premise, hosted or service-based database options that best reflect the needs of your industry and your organization. And pick a real-time business intelligence platform that supports direct connections to over 30 database and file formats, with more every year, to enable you to “pick the right tool for the job”. Choose the best mix for every project between connecting live to fast databases and importing data extracts into an in-memory analytics engine to offset the performance of slow or overburdened databases.
eBay’s data architecture comprises Teradata, Hadoop and Tableau. eBay employees can visualize insights from more than 52 petabytes of data. InformationWeek writes in December 2012 in “Big data visualization: A big deal for eBay”: “eBay uses Tableau to visualize search relevance and quality of the eBay.com site; monitor the latest customer feedback and meter sentiments on eBay.com; and achieve operational reporting for the data warehouse systems. This has helped an analytic culture flourish within eBay.”
Use the strengths of each database platform and enable them to coexist in your organization’s data architecture. Cloudera and Teradata jointly published a useful guide outlining requirements that are best suited for either a data warehouse or Hadoop.
Source: Dr. Amr Awadallah and Dan Graham, “Hadoop and the Data Warehouse: When to Use Which”, co-published by Cloudera, Inc. and Teradata Corporation. *HBase.
Big data and self-service business intelligence go hand in hand, according to by Aberdeen Group’s recently published Go Big or Go Home? Maximizing the Value of Analytics and Big Data: “Organizations with big data are over 70% more likely than other organizations to have BI projects that are driven primarily by the business community, not by the IT group.” Across a gamut of uses – from tackling new business problems, developing entirely new products and services, finding actionable intelligence in less than an hour, and blending data from disparate sources – “… big data has fired the imagination of what is possible through the application of analytics.”
Empower everyone in your organization to use and benefit from big data. Make sure people can access and interact with data in dashboards right in their browser, or on a mobile device. Because the idea that a few select people will be able to get value from big data is so 2010.
Source: Tableau Software, Mobile Business Intelligence
“Big data is indeed a big deal” said John Holdren, Director of the White House Office of Science and Technology Policy, in a March 2012 forum. Among the examples cited in a White House fact sheet: the U.S. Department of Defense is investing US$250 million annually in a series of big data programs led by the Defense Advanced Research Projects Agency (DARPA), including visual analysis of raw and unstructured data.
Consider how to integrate and blend data from disparate sources for your organization. Organizations that can blend different relational, semi-structured and raw data sources in real time, without expensive up-front integration costs, will be the ones that get value from their data. That means that you and your colleagues don’t need to know the details of how data is stored to ask and answer questions. Whether your data is in a spreadsheet, a database, a data warehouse, open source file systems like Hadoop, or all of those, you need the flexibility to quickly connect to the data you need and consolidate it.
The importance of integrating disparate big data is one area of agreement in the U.S.A. for both Democrats and Republicans. “The Obama campaign found a way to integrate social media, technology, email databases, fundraising databases and consumer market data,” said GOP digital strategist Vincent Harris, who did digital work for Newt Gingrich and Rick Perry in 2012. “That does not exist on the Republican side to that degree”, to the detriment of Mitt Romney’s campaign, quoted by Politico, “GOP seeks to up its online game”, December 8, 2012. For more on how the Obama campaign used big data, see BusinessWeek’s November 29, 2012 article “The Science Behind Those Obama Campaign Emails”.
More than 80 countries have data privacy laws. The European Union defines seven “safe harbor privacy principles” for the protection of E.U. citizens’ personal data. In Singapore, the personal data protection law takes effect January 2013. In the U.S., Sarbanes-Oxley affects all publicly listed companies, and HIPAA (Health Insurance Portability and Accountability Act) sets national standards in healthcare.
The right balance between control and experimentation will vary depending on your organization and industry. It’s never an either / or – you’ll need both – but within the constraints of your industry and your corporate policies.
If your organization has already implemented successful master data management (MDM), congratulations. But if you’re like most organizations, you’ve found agreement on definitions and business rules for master data management to be a slow, painful process for IT and business leads. A group of MIT-Sloan Management Review authors provide three tips in “Finding Value in the Information Explosion”: identify your “sacred data”; define the workflows that will use unstructured data; and use data to redefine business processes following a highly iterative process.
The contradiction of metadata is that you need to both centralize data sources and apply metadata, yet still be able to extend it by adding your own calculations; creating new groups; organizing data into hierarchies; and modifying aliases. Organizations that master big data will let IT set up the data architecture, security and access controls, while giving business people the ability serve themselves reports and dashboards.
With the backing of one or more executive sponsors, evangelists like yourself get the ball rolling and instill a virtuous cycle – the more departments in your organization that realize actionable benefits, the more pervasive analytics becomes across your organization. Fast, easy-to-use visual analytics is the key that opens the door to organization-wide analytics adoption and collaboration.
For Blastrac Manufacturing CIO Dan Murray: “Before, we were spending four to six man-hours a week producing basic reports. Now it takes us ten minutes a week. And the quality of the report is much better, much more visible and interactive.” See case studies for your industry.
So there you have it, how to succeed with big data in seven points.
Tableau Software helps people see and understand data. Used by more than 10,000 organizations worldwide, Tableau delivers fast analytics, visualization and rapid-fire business intelligence.