Underwriting AI Safety
For more than 120 years, UL (formerly Underwriters Laboratories) has been protecting consumers around the world from products that may do them harm. We’re now firmly in the digital age, but physical products obviously aren’t going away, and UL has responded by not only incorporating big data and data science techniques into its product testing routines, but also by inspecting AI and software products themselves.
UL doesn’t run the only product testing, inspection, and certification (TIC) laboratory on the planet. There are a handful of other firms that employ teams of chemists, physicists, and engineers to put products through their paces to ensure they won’t burst into flames or deliver an electric shock to its user. But Chicago-based UL is arguably the most well-known of the TIC labs, especially in the United States, which has some of the world’s toughest consumer product safety regulations in the world.
A good fraction of UL’s 13,000 employees are still involved in physically testing products to ensure that they comply with the specific product safety regulations imposed by individual cities, counties, states, and countries around the world. The company’s employees still devise all sorts of experiments to ensure the latest cell phone doesn’t inadvertently shock its users or that a hoverboard doesn’t suddenly burst into flames.
While humans remain at the heart of UL’s activities, the digital revolution is changing the testing process. In the past few years, the company has discovered that it can combine its leadership in physical sciences with emerging data science techniques to streamline the testing process in some cases, according to UL’s Chief Digital Officer Christian Anschuetz.
“Imagine you’re selling 100,000 SKUs [stock keeping units],” Anschuetz tells Datanami in a recent interview. “That’s actually a pretty small number for a large retailer or distributor. And now imagine you’re not just selling it into one little portion of California, but you’re selling it to every single city, county, and state in the United States. Now go global. Now you want to sell that same product to every part of the world.
“It’s a traveling salesman problem,” he continues. “With the nature of the products changing so quickly because of dynamic supply chains, and regulations changing at 250 times per day, you can’t employ enough people in the world to actually understand whether your 100,000 products are in compliance at all times. Humans can’t deal with that scope of data, but machines can.”
UL has had to keep pace with accelerated nature of product development. It was commonplace until fairly recently for manufacturers to digitally design and prototype a new product before moving creating physical designs and prototypes in the real world. But these days, some manufacturers are going straight from digital design and prototyping straight to product testing, skipping the physical design and prototyping stage entirely.
“That is enormous progress in an industry that would take a lot of time to evaluate a physical product, because we have to do the due diligence to make sure the products perform in accordance to the standard,” Anschuetz says. “But being able to do that electronically aids speed and aids cost. It is a huge boon for anybody that we would do that kind of work for. “
Humans in the (Big Data) Loop
UL has automated that TIC process as much as possible for the manufacturers, distributors, and retailers that pay UL for its expertise. The company hasn’t taken humans completely out of the loop. Instead, it augments humans’ innate ability to pinpoint solutions with machines’ capability to spot anomalies hiding in a sea of data.
“Machines can’t do everything,” Anschuetz says. “But we can help the humans narrow in on the areas that are highlighted and then really make sure that any problems or issues are mediated at the highest level.”
UL brings a range of data types and technologies to bear on this challenge. One of the biggest challenges is that most of the data that UL uses exists in unstructured formats, such as Microsoft Word documents and Excel spreadsheets.
“It’s almost entirely unstructured data,” Anschuetz says. “We have a group of people who are really good at taking unstructured data and…creating enough context so that the machines can then create relationships between the data sets.”
UL stores this “amorphous blob” of data (as Anschuetz adeptly puts it) in a mixture of cloud-based data lakes, including Amazon‘s S3 and Microsoft’s Azure BLOB store. Once the data relationships have been defined, the for-profit company (it was a non-profit organization until 2010) uses a mix of open source and proprietary programs and algorithms to determine whether a given product has satisfied safety and sustainability requirements in specific jurisdictions. Hadoop and TensorFlow are utilized for some of this processing, as are Oracle‘s Exadata machines, as this YouTube video can attest to.
While UL relies on the big data machines to do much of the heavy lifting these days, in the end, the workflow always comes back to human workers.
“We’re letting the machines collect the information and then we sift through it, run it through colander of sorts, so the experts can … get an insight and apply their judgement for the insights,” Anschuetz says. “They might even help the machine learn by saying, yes that it correct, or no it’s not and here’s why. But it comes down to people. This is a people business.”
Summoning AI Tests
Recently UL has started expanding the types of products that it tests in response to the proliferation of digital services on mobile phones and the Internet.
“We employ ethical white-hat hackers to make sure that mobile payments are conducted, say on a phone, in the most secure fashion,” Anschuetz says. “Our definition of safety has changed. You can’t really live in a safe world if can’t livein a financially secure one, right?”
The company has even started looking into testing AI products. Lots of people consider AI a threat to humanity, including Elon Musk, the Tesla founder who once compared efforts to build AI to “summoning the demon.” While UL’s AI-testing efforts are still nascent, you can credit Musk himself with bringing UL to this particular table.
“Elon Musk often talks about AI. He has concern about AI,” Anschuetz says. “In one of his discussions online and on Twitter…he called on UL to bring together academia, government, and industry, and convene the minds and the bodies together to say ‘How do make sure we use artificial intelligence in a way that does more good than harm.’ We heard the call and there’s a number of things that we’re investigating right now.”
While Anschuetz didn’t share any specifics about the ways that UL will work to ensure the safety of consumers in an AI world, he did indicate that such a challenge was firmly in the realm of what a modern and digitized UL can do for humanity.
“There are so many powerful and incredibly awesome advances that are happening every day, and yet there is a need for companies like ours to make sure that when these things come in, that somebody is looking at it and saying, how do we also make sure that they don’t go astray and do harm,” Anschuetz says. “That’s our purpose. We’re very passionate about that. We want AI. We want all these technological advances to come and benefit society, and we also want to avoid any of the pitfalls that we have seen in societies throughout the history of mankind when we just weren’t as thoughtful, as a country or a community, as we should have been. That’s our job to ferret it out — to make sure that you’re safe, your kids are safe, and that we’re all safe and living in a sustainable world.”