AMEX Adopts Machine Learning to Crunch More Data
Machine learning continues to make inroads in the financial services sector and other large enterprises as businesses seek to leverage the technology to get a better handle on customer data and preferences.
A user and vendor recently described a large-scale machine-learning project that deployed the technology in a production-ready big data platform. Details were provided during a MapR presentation at the Hive Big Data Data Think Tank.
The user, American Express, looked to machine learning after realizing that traditional databases could no longer handle the volumes and velocity of data and breadth of analytics needed to deliver financial services. The company initially rolled out a Hadoop platform; it then sought to apply machine-learning techniques as a way to sift through financial transactions and other AMEX services.
The credit card company also turned to database leader MapR to get a handle on soaring customer data volumes increasingly coming from mobile devices and the company’s web site. The move to machine learning was prompted by the realization that the ability to leverage customer data would give it a leg up over competitors in the cutthroat credit card business.
Among the use cases implemented by American Express are fraud detection and prevention, attracting new customers in a zero-sum market where consumers are wary of taking on more debt and, third, improving customer experience.
Machine learning was used to improve fraud detection through modeling methods that used a range of data sources including merchant information and spending details to spot possible card fraud and alert card holders. Machine learning helped to detect suspicious activity sooner while allowing AMEX to scour a larger data set to spot fraud and alert customers in milliseconds, the company said.
The machine learning techniques met the company’s detection and split-second decision-making requirements, ultimately outperforming traditional linear regression methods.
American Express previously generated about 90 percent of new customers via scattershot and seemingly never-ending direct mail campaigns. New online services along with targeted marketing through machine learning models now account for about 40 percent of new customers. Moreover, online customer acquisition has proven much cheaper than relatively expensive direct mail pitches.
Machine learning applications place unique requirements on data analytics platforms. Hence, MapR’s platform was geared to digest large volumes of data from variety of sources. These data were prepared and staged so that AMEX could use machine learning to squeeze out as much useful information as possible, the database vendor stressed.
MapR also emphasized the need to store, stream and facilitate search functions on large and fast-moving data sets. Its approach to machine learning also relies on a real-time read/write file system, an integrated NoSQL database and an array of Hadoop tools intended to handle large-scale machine learning applications like the AMEX example.
The database specialist’s approach is also able to make direct use of legacy code to generate consistent snapshots of data versioning. It also relies on remote mirroring for applications synchronized across multiple datacenters, MapR said.