Columbia Business School Researchers Argue Future of Big Data Lies in Its Ability to Assess Consumers’ Mindset in Real Time

NEW YORK, July 31, 2018 — Marketers have long relied on demographic information to identify, get to know, and predict the behavior of their customers. With the rise of big data, marketers not only have the opportunity to better predict consumers’ behavior, but also to understand and respond to their immediate psychological needs, according to Columbia Business School researchers.

In a recent discussion paper for Current Opinion in Behavioral Sciences, Columbia Business School’s Sandra Matz and Oded Netzer address the challenges and opportunities of using big data to benefit both business and consumers based on psychological profiles drawn from information posted on personal websites and discussion forums, and language used on Facebook and Twitter.

“Big data usage is quickly evolving,” said Oded Netzer, Professor at Columbia Business School. “With technological advances in the collection, storage and analysis of large amounts of data, businesses can now gain valid insights on millions of consumers as they go about their daily lives.”

Big Data as a Window into Consumer Psychology

In the age of big data, psychological traits including personality, IQ and political orientation can be accurately predicted from consumers’ digital footprints. The combination of information about ‘what one does’ with a deeper understanding of ‘who one is’ offers tremendous opportunities to not only boost the effectiveness of marketing campaigns, but also to help consumers make better decisions.

“One benefit of psychological profiling is that the pre-selection of ads based on psychological needs can alleviate the problem of choice overload,” said Sandra Matz, Assistant Professor at Columbia Business School. “It can even help target highly neurotic individuals who display early signs of depression with ads that guide them to self-help pages or offer professional advice.”

Alongside the benefits, however, this more personalized marketing raises new ethical challenges. With the introduction of even more sophisticated prediction algorithms that not only analyze individual behaviors, but also make inferences about a consumers’ intimate psychological traits, the issue of privacy concerns is paramount. For companies like Facebook, who are looking to alleviate these concerns and change negative perceptions of data collection, the researchers say showing the value of the data they are looking to collect from consumers is key.

Ultimately, the ability to predict consumers’ psychological traits and states through big data offers exciting new opportunities for digital marketing. In the next 10-15 years, the researchers predict that the integration of this data into one, comprehensive consumer profile will result in tailored marketing campaigns that are responsive to the anticipated needs and desires of consumers in real time.

To learn more about the cutting-edge research being conducted at Columbia Business School, please visit www.gsb.columbia.edu.

Source: Columbia Business School