Leveraging Big Data to Economically Fuel Growth

In conjunction with MapR, Datanami presents TransUnion with this month’s “Big Data All Star” award.

Kevin McClowry has been working with Big Data even before the term hit the mainstream. And these days, McClowry, currently the lead architect over analytics with TransUnion, LLC, is looking to Big Data technologies to add even more impetus to his organization’s growth.

“My role is to build systems that enable new insights and innovative product development,” he says. “And as we grow, these systems need to go beyond traditional consumer financial information. When people hear TransUnion, they immediately think of their credit score – and that is a huge part of our business. What they don’t realize is that we also have been providing services across a number of industries for years – insurance, telecommunications, banking, automotive to name a few – and we have a wealth of information. Enabling our analysts to more effectively experiment within and across data domains is what keeps that needle of innovation moving.”

But growth is rarely achieved without combatting some degree of inertia. “Within most organizations, it is the customer facing, mission critical systems that warrant the financial investment in ‘enterprise-class’ commercial solutions. In contrast, R&D environments and innovation centers tend to receive discretionary funding. We started down this road because we knew we wanted to do something great, but we were feeling ‘the pinch’ from our current technology stack,” McClowry says.

Data to the People

“The first problem I wanted to address was the amount of time our analysts spent requesting, waiting on, and piecing together disparate data from across the organization. But there was a lot of data, so we wanted to incorporate lower cost storage platforms to reduce our investment.”

So McClowry went to a hybrid architecture that made use of commercial databases for the most desirable, well-understood data assets – where the cost can be justified by the demand – surrounded by a more cost-effective Hadoop platform for everything else. “The more recent, enterprise data assets are what most people want, and we’ve invested in that. But the trends in historical data or the newly acquired sources that don’t have the same demand are where a lot of the unseen potential lies. We put that data in Hadoop, and get value from it without having those uncomfortable conversations about why our analysts need it around – most of us have been there before.”

Since adopting this new “tiered” architecture, McClowry’s organization has seen the benefits. “We’re seeing analysts work with data in a way that was previously out of reach, and it’s fantastic. When the new normal for our folks starts to include trillion-row datasets at their fingertips…that’s fun.”

New Tools, Enabling Better Insights

The next step for McClowry’s team was to enable tools that would make that data accessible and usable by the company’s statisticians and analysts.

“We’re finding that a lot of the Data Scientists we’re hiring, particularly those coming out of academia, are proficient with tools like R and Python. So we’re making sure they have those tools at their disposal. And that is having a great influence on our other team members and leading them to adopt these tools.”

But that was not the only type of analyst that McClowry’s team is looking to empower.   “I’m consistently impressed by the number skilled analysts we have hidden within our organization that have simply been lacking the opportunity to ‘fall down the rabbit hole.’ These are the people that have invaluable tribal knowledge about our data, and we’re seeing them use that knowledge and data visualization tools like Tableau to tell some really powerful stories.”

Hitting the Road

McClowry’s team is globalizing their Big Data capabilities, introducing the analytics architecture worldwide and sizing it to fit the needs and resources of each country’s operation. In the process, they are building an international community that is taking full advantage of Big Data – a community that did not exist before.

“We are trying to foster innovation and growth. Embracing these new Big Data platforms and architectures has helped lay that foundation. Nurturing the expertise and creativity within our analysts is how we’ll build on that foundation.”

And when asked what he would say to other technologists introducing Big Data into their organizations, McClowry advises, “You’ll be tempted and pressured to over-promise the benefits. Try to keep the expectations grounded in reality. You can probably reliably avoid costs, or rationalize a few silos to start out, and that can be enough as a first exercise. But along the way, acknowledge every failure and use it to drive the next foray into the unknown; those unintended consequences are often where you get the next big idea. And as far as technologies go, these tools are not just for the techno-giants anymore and there is no indication that there will be fewer of them tomorrow. Organizations need to understand how they will and will not leverage them.”