Follow Datanami:
November 11, 2015

10 Tips for Beginning Your Big Data Journey

If your organization is just starting your big data analytics journey, don’t fret: You’re not alone. Plenty of others are in the same boat. However, catching up to the analytics leaders will take a lot of hard work.

Here are 10 big data tips, which were culled from discussions at recent industry events, including Teradata’s PARTNERS 2015, SAS Analytics 2015, and the CAO Summit.

1. Avoid Shiny Objects

Organizations have access to more data than ever, from sensors to smart phones, and the pressure to do something big is bigger than ever. It’s easy to get caught up in the hype, especially around shiny new technologies.

But finding success with big data is not easy, and it’s not something that can be done overnight by simply adopting technology. Today’s big data leaders have been working at analytics for 10 years or more, and already have the foundations in place that lead to success.

“A lot of times, laggards look at silver bullets,” says Ron Bodkin, the founder of ThinkBig, a data analytics services firm bought by Teradata last year. “They hope they’ll adopt a technology and all their problems will be solved, which of course never happens.”

2. Don’t Worship Data


(image courtesy grace21/

Data, it’s been said, is a new “currency” that has implicit value in of itself. That may be taking things too far, especially within companies that are measured by their bottom line. Instead of hording data like a greedy, gold-loving dragon, collect only what you need to answer business questions.

“Data in of itself can be an competitive advantage, or you can have analysis paralysis,” says Matt Ariker, chief operating officer of the Consumer Marketing Analytics Center at McKinsey. “I’m old. I started at P&G and we would spend 12 weeks analyzing a two-week promotion.  We never got anywhere. You really need to think about what are those high value questions and how do you integrate structured and unstructured data, and sort through how do you automate what you’re going to do with the insights.”

3. Put the Business Case First

Some companies beginning their analytics journey make the mistake of collecting all the data they can get their hands on, putting in a big data lake, and hoping a magical algorithm will deliver business insights at the press of a button. That approach rarely gets you anywhere.

“People have this misnomer that what data scientists do is come into the office on Monday and say ‘What fun thing am I going to work on with no constraints?'” says Bodkin. “I don’t know a lot of organizations that have a large budget for unconstrained explorations.”

4. Establish an Analytics Culture

You can have the world’s best algorithms working on the cleanest data imaginable to create amazing insights, but it won’t do any good unless your business folks have bought into the value of analytics and trust the data and the insights. That requires building an analytics culture.

“If you look at the most advance analytics competitors out there they have spent the last 10 to 15 years transforming themselves,” says Oliver Ratzesberger, the president of Teradata Labs. “There are companies that say they need to go from crawling to sprinting [on big data projects] …in the next 90 days. It’s not going to happen. It’s almost irrelevant what the technologies are. It’s about this big cultural transformation.”

5. Fail Fast, But Succeed Eventually


(courtesy Karen Road/

Data science is an iterative process that typically involves some degree of failure before hitting upon the right recipe for turning data into actionable insights and operationalizing them. Many of the recent advances in the big data field, such as the advent of Apache Spark, are focused on accelerating that process.

But big data practitioners should not be caught up in iteration for iteration’s sake. “The fact that you can test a lot of things and fail faster actually just means you failed faster,” says McKinsey’s Ariker. “In the end it’s got to be about business-backed, hypothesis-driven execution of agility. This is about giving better results, not just learning for agility’s sake.”

6. Keep C-Suite In the Loop

Sharing your big data success with senior management is necessary, not just to ensure they don’t pull the plug on your future big data projects, but to ensure that you’re asking the right questions.

“One of the things we learned is you have to continue to drive value all the way to that C-suite,” David Dittmann, the director of business intelligence and analytics services and Proctor & Gamble, said at the recent CAO Summit. “If you’re not there when they’re discussing all the business questions., I think you’re fundamentally going to be working on the wrong thing.

7. Governance Is Boring But Important

There’s no quicker way to inducing a late afternoon nap than an in-depth discussion of process and change management. But in the fast-paced world of big data, keeping a handle on all the moving parts isn’t just a nicety—it’s critical for long-term success.

It’s important to be agile enough to try something for 30 days, Teradata’s Ratzesberger says. “But you need to have a foundation in place that’s allows you to do that, yet is integrated enough that all this governance—lineage and error handing and version control–must happen,” he continues. “Something may work for the first 30 or 90 days, but a year from now you have a house of cards and you pull one wrong card and it all comes crashing down.”

8. Think It All the Way Through


(courtesy antoniodiaz/

You may have the best predictive model, but unless it can be operationalized in the real world and have a positive impact, then it really has no value to the organization and is a waste of time and resources.

“If you go to manager and say ‘Good news, we have lots of productivity out of our big data platform but I need to hire 1,500 more people,’ you’ll probably be looking for a new job the next day,” Kinsey’s Ariker says. “You have to think it all the way through to automation and impact, as well as how do you make sure you’re answering the high-value question and you can do something with the answer.”

9. Less Can Be More

In big data, it can be tempting to instrument everything. When you’re collecting so much data about various parts of the business, and the algorithms are kicking out decent insights, you want to instrument business actions around lots of variables. But that approach often doesn’t fly.

Jennifer Lewis Priestley, a professor of applied statistics and data science at Kennesaw State University, tells the story of a math major who creates a terrific model. It has a very high degree of accuracy, and generates 2,500 predictors. “It makes no sense,” she said at the recent SAS Analytics 2015 conference. “You can’t operationalize 2,500 predictors. So I tell the math major to go find four that I can put into practice.”

10. Don’t Go Hunting Unicorns


(courtesy lyeyee/

Data scientists who are well-versed in statistics, technology, and business have been called “unicorns” because they’re so rare. (Actually, no unicorns exist because they’re mythical creatures, but that’s another story.)

While such data scientists do exist, it’s not worth your time trying to lure them into your organization, says ThinkBig’s Bodkin. “All our customers are struggling with this,” he says. “You don’t get that from one person. You get that from teams–high functioning data science teams deliver results.”

Related Items:

Exposing the Data Scientist Myth: Using Big Data Without Them

Data Science Education Gets Stronger, But It’s Not There Yet

One Deceptively Simple Secret for Data Lake Success

(feature art courtesy SunnyStudio/