Follow Datanami:
November 19, 2014

Why Twitter Is the Low-Hanging Fruit of Social Analytics

Nobody is mistaking the Twitter fire hose for a crystal ball. But big companies are increasingly turning to Twitter and other social media outlets for a range analytic tasks, from gauging customer sentiment about bendy smartphones to saving costs during product recalls.

Corporations have been analyzing social media since it arrived on the scene year ago. But only recently have they figured out how to really make use of it, according to Katherine Matsumoto of Attensity, a big data analytics firm based in Salt Lake City.

“It’s really an under-tapped data source,” says Matsumoto, a natural language processing (NLP) expert. “We like to think of social media as being able to harvest the universe’s biggest consumer focus group. You can ask and answer any question you need.”

Attensity has parlayed its background in providing text analytics for big companies and the government into becoming a one-stop shop for social media analysis for the likes of Ebay, Yahoo, Microsoft, Verizon, and Whirlpool. The company provides a soup-to-nuts solution that lets non-technical users ask questions of the uber focus group that is social media and see results in real time. It exposes to its customers more than 550 million social media posts and tweets per day, including the entire Twitter fire hose via a contract with Gnip (now part of Twitter proper).

The company has done a lot of the standard social analytics fare, including answering questions such as “Does my new marketing message resonate with customers?” But Matsumoto has seen other more interesting use cases where  companies find value in other ways, including identifying instances of fraud.

“We work with insurance companies that are working on cases where they can identify potential connections between product recalls and claims they’ve had to issue, where they might be able to avoid responsibility for paying for incidents that can be traced back to a recall,” she says. “That kind of information is just circulating.”

Even IBM is warming up to the low-hanging fruit that Twitter provides the world. Just last month, the century-old IT behemoth formed a partnership with Twitter whereby it will pipe the fire hose into enterprise accounts and help them mine it for useful data. IBM CEO Ginni Rometty applauded Twitter for creating “something extraordinary” and being able to “take the pulse of the planet.”

Hooking up with Twitter is a smart move on IBM’s part, says R “Ray” Wang, principal analyst and founder of Constellation Research.

“I think we’re starting to determine correlations from social signals,” Wang tells Datanami. “What IBM and other analytical players can do is identify potential linkages that can help predict next best actions. It takes a lot of data to crunch and it’s still possible for false positives and false negatives to emerge. However, businesses should treat these as strong signals to decision making not as [an end-all, be-all]. We still have to connect other data points and determine how strong these signals are.”

Another member of IT’s old guard that’s now embracing social analytics is FICO. Formerly Fair Isaac, the firm best known for building the tools for determining consumer credit scores last week announced it will be delivering a text analytic tool next quarter that can mine Twitter and other social properties for useful information.

“Enterprises are waking up to its significance and the technologies are getting easier to use,” says FICO’s Big Data Strategist Martin Hall, the founder of Karmashere, the Hadoop application developer that FICO bought earlier this year. “It’s clearly mainstream.”

While FICO didn’t divulge much about the forthcoming text analytics tool, the goal will be to enable customer to mine social media sites like Twitter and Facebook, as well as other sources, such as email. Given FICO’s expertise in the finical services business, it could also harness social media data to improve services it’s already providing customers, including fraud detection and determining credit scores.

“Twitter and Facebook are obviously good examples of semi-structured or unstructured data that can be harnessed in this new world of big data, where the platform itself, namely Hadoop, makes it possible and products like ours make it much easier for the practitioners–the data scientists, the analysts, and the business users–to actually go at that data, whatever it is,” Hall tells Datanami.

Much of the focus remains around customer sentiment, which is the original use case for social media leveraged by brand marketers. That means identifying positive and negative sentiment around not only your company, but your competitors. “If you can discover the negative sentiment and messages around competitors’ products, and you can combine that with positive sentiment about your own products, then that can inform the messages you use to target potential customers based on feedback from the marketplace,” Hall says.

Given the volumes of data, however, one must approach social media analytics somewhat carefully. There are 500 million Tweets made per day, but the vast majority of them will be worthless to you and your company.

Just the same, given the volume, there are jewels hidden in there available for the taking, says Mike Stringer, chief data scientist and co-founder of Datascope Analytics, a big data consulting firm based in Chicago.

“It is loads of crap, and the fire hose is so big that the fraction of non-crap is pretty small,” Stringer says. “But the absolute amount of non-crap is still a lot of good stuff. It’s the classic information retrieval problem around needles in the haystack. And with Twitter and the Internet, the haystack is really, really big, so there are a lot of needles.”

Related Items:

Twitter Funds MIT ‘Social Machines’ Effort

IBM Embraces the Twitter Fire Hose

Twitter Looks to Leverage Social Data with Gnip Deal