February 6, 2012

Financial Firms See Big Data’s Big Picture

Robert Gelber

The recent boom in data-generating tech, the wave of new financial regulations on the horizon, and competitors constantly upping the advanced analytics ante has financial institutions looking for answers to their big data problems.

The current data situation for a lot of firms is like having sections of a jigsaw puzzle in different rooms, but the puzzle keeps growing and the main sections have not been put together. This is a problem that many institutions are facing right now; they are collecting massive amounts of data, but aren’t always sure what to do with it, or where the value is… or even how to store it efficiently.

So the first step, it seems, is to put the puzzle together—something that NoSQL and Hadoop approaches are helping financial services firms do better. One of the main advantages Hadoop-based solutions have over relational database applications is their ability to easily link dissimilar sources of data.

But even post-Hadoop, with all the pieces of the puzzle in the same room, something needs to happen for them to still fit together before the big picture emerges. Once all the information is in one place, the final step is to use analytics to make proper use of all that captured data.

If firms glean valuable information before their competitors do, they get the upper hand. This is a two way street, however, as competing firms are using advanced analytics to incentivize customers away from their current financial institutions.

In an interview with Bank Systems & Technology, Allen Weinberg, director of business technology for McKinsey & Co. said “in terms of information that someone wants to go after. In some ways drawing the value out of the data has the problem of increasing the value of the data, which makes it more valuable as a target.”

Not only is this technology useful for profits and productivity, it’s becoming a requirement as new financial regulations are on the horizon. Legislators have been, and continue to advocate for better risk analysis from the nation’s financial institutions. Firms using high performance computing to gain faster results from the stock market might verify their forecasted risk is as accurate as possible.

Caution must be taken when deciding to implement Hadoop-based solutions. Hadoop has the potential to be very powerful, but it should not be seen as a complete replacement to a firm’s current IT infrastructure. Many firms have successful systems currently in place and new analytics solutions could be seen as a means to provide relevant information to their arsenal.