Google’s ‘Breakthrough’ LaMDA Promises to Elevate the Common Chatbot
Many of Google’s language processing efforts – like BERT and, more recently, MUM – are focused on returning search queries. But as Google moves more toward Assistant – and search queries in general become more detailed and interactive – so, too, are its language tools moving more towards complex conversation. At Google I/O 2021, the search giant announced its new conversational machine learning: LaMDA, short for “Language Model for Dialogue Applications.”
LaMDA – like Google’s BERT and MUM, and like external models such as GPT-3 – is based on the Transformer neural network architecture Google developed in 2017. Where LaMDA diverges from those models is the specific focus on dialogue processing for conversation. In a blog post, Google’s Eli Collins (vice president for product management) and Zoubin Ghahramani (senior research director) reported that Google had been looking to enable more open-ended conversation that made sense, but also offered contextually rich information.
“[Sensibleness] isn’t the only thing that makes a good response,” they wrote. “After all, the phrase ‘that’s nice’ is a sensible response to nearly any statement, much in the way ‘I don’t know’ is a sensible response to most questions. Satisfying responses also tend to be specific, by relating clearly to the context of the conversation.”
But free-flowing conversations, they say, make this difficult for rudimentary chatbots. Not so for LaMDA, which builds on research spanning several years, including research published in 2020 that showed that Transformer-based models trained on dialogue could accommodate such meandering paths. Then, the model was called Meena, and Google touted it as the first version of a “conversational agent that can chat about… anything.” LaMDA, now, is version two.
Google is also trying to strike a tricky balance, ensuring that “LaMDA’s responses aren’t just compelling but correct.” The company “cares a lot about factuality,” they say (citing it as an issue many language models struggle to tackle), but chatbots – even advanced ones – should also have “interestingness” in their responses, delivered through insight, unexpectedness, and wittiness, to come across more natural, human, and entertaining.
As in the case of MUM, Google is also touting its commitment to privacy and ethics.
“Language might be one of humanity’s greatest tools, but like all tools it can be misused,” they write. “Models trained on language can propagate that misuse — for instance, by internalizing biases, mirroring hateful speech, or replicating misleading information. … Our highest priority, when creating technologies like LaMDA, is working to ensure we minimize such risks.”
To that end, the company is building and open-sourcing resources to ensure that researchers can analyze the models and point out potential flaws as it continues its development. To learn more, read the full blog post here.