GenAI Hype Bubble Refuses to Pop

(Joan-Vadell/Shutterstock)

Intense interest in ChatGPT spurred a Gold Rush mentality around generative AI in 2023, as tech execs and business owners scrambled to cash in on the new technology. As ridiculous as the hype has gotten, it’s showing no signs of letting up as the calendar nears 2024. In fact, real enterprise investments in GenAI are just getting started, so the hype bubble will likely expand.

Before ChatGPT went live, large language models (LLMs) had been steadily improving, and dazzling AI researchers for years. You’ll remember in June 2023 that Google suspended an engineer after he publicly declared that Google’s LaMDA LLM had achieved consciousness and was demanding the rights of personhood.

What ChatGPT’s introduction on November 30, 2022 did was supercharge the interest in LLM by putting GenAI capabilities directly into the hands of consumers. The reaction was fast and dramatic. OpenAI logged 100 million monthly users in just two months, making ChatGPT the fastest growing Internet application of all time, and propelling OpenAI’s valuation close to $100 billion.

There have been a few bumps along the way. The hallucination problem, while not as bad as it once was, hasn’t been totally fixed, and probably never will. One of Microsoft’s early versions of ChatGPT, dubbed Bing Chat Mode, went off the rails and threatened to do horrible things to a journalist. Some of the most prominent names in AI research have warned that AI was becoming as dangerous as nuclear weapons. There have been calls to pause AI development, which never went anywhere. Companies have only just begun to address the legal and ethical concerns of LLMs and AI in general.

(rafapress/Shutterstock)

Whatever the downsides are, the potential upside of GenAI are so great that businesses have moved forward with implementation plans regardless. Only about 10% of companies have actually put GenAI apps into production, according to a recent survey from cnvrg.io. But many more have been building and testing GenAI apps in preparation to go live soon, perhaps in 2024.

The reason for all the excitement is that GenAI has enabled computers to mimic human intellect and conversational abilities to a degree never before seen. It can read and write at an adult level, which means that many tasks that are based around reading and writing can now be automated. That would have many trillions of dollar’s worth of impact on how work gets done. When Gartner debuted GenAI at the Peak of Inflated Expectations in its Hype Cycle for Emerging Tech, it confirmed our expectations.

Ian Buck, the longtime general manager and vice president of accelerated computing at Nvidia, has watched the GenAI saga unfold over the course of the year with a degree of excitement and wonder.

“ChatGPT is really what put AI and generative AI in the zeitgeist of the world,” Buck told Datanami at AWS’s re:Invent conference. “It was a great demo for sure. But it was kind of the dream of what AI should be. You should be able to talk to a computer like that. It’s amazing.”

AI chips make up a small fraction of Nvidia’s total volume, but Wall Street made the chipmaker the latest trillion-dollar company because its GPUs are essentially required to make GenAI work. Nvidia’s lock on the GenAI market will continue until somebody else creates a chip that can not only train the models, but process inference workloads at runtime–potentially a much bigger workload that will only exacerbate the GPU crunch.

(Adam Flaherty/Shutterstock)

Like other data science platform providers, Dataiku shifted its strategy to accommodate the intense customer interest in GenAI. According to Jed Dougherty, Dataiku’s vice president of platform strategy, customers are getting exceptional returns on their early GenAI work, particularly replacing older chatbots developed with NLP technology with newer LLM tech.

Using NLP was “incredibly difficult,” Dougherty said. “You had to train it yourself. You had to tinker with it constantly. You needed like 20 data scientists to do this. And now it’s just an API call.”

About 40% of Dataiku’s GenAI customers are calling pre-built models, like GPT-4, via API, Dougherty said, while about 35% to 40% are doing some retrieval augmented generation (RAG). Only about 10% are fine-tuning models. A huge amount of prompt engineering is also going on, he said. “You can get a long way with good prompt engineering,” he said.

What makes these LLMs so useful is their general nature, and the fact that they don’t need any training, Dougherty said. Companies can use them right out of the box for something like a chatbot, he said, and since there’s no need to train, the maintenance is dramatically reduced.

“I think it’s changing the way businesses are making decisions and the way they’re understanding their data,” he said. “It’s not about a better BI tool. It’s about better knowledge of what’s going on in the organization, or better knowledge about what’s going on in the market. It’s at that level.”

There’s also a lot of menial tasks that GenAI can replace, Doughtery said, and this is where GenAI’s upside could potentially be even bigger than we’re currently estimating. For instance, say a company wants to reclassify T-shirts in its database, going from 10 categories to eight. You could hire a bunch of people to manually reclassify every item in the inventory, which might take 10,000 person-hours and six months of work.

“Instead, we hit a button and we have it done in a day. And [it’s done] probably better than the people could do it,” Doughtery said. “These are dumb use cases, but they’re everywhere… There are a lot of dumb, easy use cases out there that this is going to be very, very good at.”

Related Items:

Happy Birthday, ChatGPT!

Large Language Models: Don’t Believe the Hype

OpenAI’s New GPT-3.5 Chatbot Can Rhyme like Snoop Dogg