Follow Datanami:

2018 – GDPR and the Big Data Backlash

If the first seven years of Datanami’s existence were primarily about the rapidly expanding use cases for big data, advanced analytics, and AI, then the eighth year marked the first major pullback in the use of these technologies.

2018 wasn’t the first time that we heard howls against big data abuses, privacy violations, and lapses in the ethical use of AI. But one can make the argument (as we are doing here) that 2018 was the year that these concerns began to reach a critical mass in the public space.

The Cambridge Analytica scandal got the ball rolling in March 2018, when news broke about the improper way the analytics company used data from Facebook during the 2016 presidential election. Soon, people across all spectrums of society were questioning how their data was being used college admissions, how police departments use it to inform patrol patterns, and how banks use it to approve mortgages.

The General Data Protection Regulation (GDPR) went into effect on May 25 2018, providing residents of the European Union with a slew of new digital rights. Companies around the world suddenly needed permission to obtain personal information about their customers, could be punished if inadvertently lost it, and had to abide by individuals “right to be forgotten.”

Many companies fretted that this would spell the end of big data and AI as we know it, and they were right. GDPR spelled the end of big data’s Wild West period, and marked the beginning of a new era of heightened responsibility, not only in the handling of data, but in the use of AI. It also helped spur passage in July 2018 of the California Consumer Privacy Act (CCPA), which would bring a GDPR-like law into effect in 2020.

Data governance jumped off the backburner and roared to life as a key element in the responsible use of big data. While big data cowboys were used to moving fast with little to no accountability, it suddenly became cool to exhibit full control over one’s data, which bolstered the market for tools like data catalogs, metadata management tools, privacy and security tools, and data quality tools. The message became clear: When customers have faith in how a company handles customer data, that company gain a competitive advantage over sloppier companies.

At the same time, questions began to emerge about just how those deep learning models actually work. In many cases, the exact mechanism remains a mystery. Various explainable AI techniques emerged to demonstrate how neural networks can be deployed without violating ethical guidelines, including bias against underrepresented minorities. In the end, many companies in regulated industries have resorted to using traditional machine learning techniques, which are less accurate but more easily explainable.

Amid the big data abuses and black box neural networks, another anti-AI thread wove itself through 2018’s tapestry: the impact of AI on jobs. A Pew Research poll found nearly three-quarters of Americans worried about robots taking their jobs. Indeed, Chinese AI expert Kai-Fu Lee wrote a 2018 book titled “AI Superpowers” in which he predicts that AI would eventually displace 40% of the world’s jobs.

The events of 2018 represented the first major pullback in what had been fairly rapid and linear adoption of ever-improving technology. The corporate boardroom suddenly had to balance the potential downsides of using big data, advanced analytics, and AI–including sizable fines and reputational harm—against the direct benefits to the company’s operations and its bottom line.

The digital well-being of individuals and consumers was suddenly front and center, and it hasn’t really left that position thanks to the proliferation of GDPR-like laws around the world. Meanwhile, the world continues to grapple with difficult issues, such as the use of facial recognition technology that has been shown to work poorly on ethnic minorities. Big Tech CEOs would be brought before Congress in July 2020 to be questioned about their data-fueled business practices, and a film called The Social Dilemma would highlight some of the most egregious abuses. But the anger would continue to build as the tech giants used their mastery of big data, advanced analytics, and AI tech to expand their reach into the world’s consumer markets.

Related Items:

2020 – COVID-19 — Kicking Digital Transformation Into Overdrive

2019 – DataOps: A Return to Data Engineering

2017 – AI, Deep Learning, and GPUs

2016 – Clouds, Clouds Everywhere

2015 – Spark Takes the Big Data World by Storm

2014 – NoSQL Has Its Day

2013 – The Flourishing Open Source Ecosystem

2012 – SSDs and the Rise of Fast Data

2011 – The Emergence of Hadoop