AI Bubble Goes Boom
And the tech world holds its breath.
If you stop for a minute, shut your eyes and listen in very closely, you can almost hear a clackety-clack of a keyboard as an AI Guru drafts his next tweet to his 17 followers on how you can reach $323472 MRR in 5 easy steps.
All by leveraging ChatGPT, a money-printing machine that feels illegal to know.
Those self-proclaimed experts are steering us at breakneck speed towards the inevitable burst of the AI bubble, and I cannot wait to see it happen.
Of Bulbs and Bubbles
As a society, we do not have a very good history of not blowing things out of proportion - we invest in Beanie Babies, get ourselves Pet Rocks and buy digital art of Bored Apes we technically own, but not really (I took a screenshot of a few, you can sue me).
In 1637, a bulb of a tulip known as ‘the Viceroy’ was offered for sale for between 3000 and 4200 guilders. The same price as you would pay for buying a very decent house.
Carolus Clusius was the botanist who pioneered the cultivation of tulips in the Netherlands. His fascination with the flower led him to grow a private garden with many rare types and spend his later years studying the mysterious phenomenon called ‘tulip breaking’.
Tulip breaking is when a tulip petal colour changes into a multicoloured pattern. Science later found out that the breakages were caused by a virus infection.
People in Holland and the rest of Europe were obsessed with the infected tulips, much more than they admired the normal ones. This ramped up their price, and many Dutch botanists competed with one another to breed new and more beautiful tulips, known as ‘cultivars’.
The mesmerizing beauty of multicoloured petals and the rise of the independently rich Dutch traders turned tulips into a fashionable trading asset (you see, the difference is a tulip is far prettier than an NFT).
Due to the exaggerated expectations of profit returns, the tulip bulbs became the fourth leading traded item in the Netherlands, after gin, herrings, and cheese.
As more people started buying them as an investment, the price went up further, until people were buying tulips because the prices always went up because people were buying tulips because the prices always went up etc.
That until on February 7th 1637, almost overnight, the price structure for tulips collapsed, sweeping away fortunes and leaving behind financial ruin for many ordinary Dutch families.
It started in Haarlem when people refused to show up to collect the tulips they ordered, partly due to the recent outbreak of bubonic plague, then spreading to other parts of the Netherlands and Europe.
Today, we'd call it 'investment diversification' and write a think-piece about it on LinkedIn. But back then, it was just a big economic bubble called the Tulip Mania.
And we have experienced quite a few of those in the recent decades.
Amar's Law and Hype Cycles
We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run.
Roy Charles Amara
Remember the dot-com boom of the late 90s? Everyone was convinced that every new website would turn into a goldmine overnight. The top 10 stocks had a ratio of 24 times the amount of value compared to their actual earnings. Stock prices went over the roof, only to come crashing down when the reality hit.
Yet, from that rubble, the true giants of the internet era - think Amazon and Google - emerged and reshaped our world.
The cycle is predictable:
First comes the hype, where everyone believes this new tech will solve all problems, cure all diseases, and possibly walk your dog. Investments pour in, and everyone wants a piece of the action. Prices skyrocket as fast as the promises.
Then reality kicks in. The technology doesn’t deliver instant miracles, investors get cold feet, and the bubble bursts. Prices drop, and those who bought into the hype are filing bankruptcy. It’s the classic rise and fall, as history loves to repeat itself.
And then comes the twist: while we’re busy looking for someone to blame, the real value of the technology quietly starts to unfold. The internet, once a bubble, is now an essential part of our lives.
In the classic Gartner Hype Cycle that observes how new exciting technologies enter the market, the stereotype is for the hype to get out of control, before falling back down to earth, slowly reaching a level of maturity.
The New Sheriff in Town
And then comes Artificial Intelligence, the new sheriff in town. The term was coined already in 1955 by John McCarthy, but the actual hope, hype, and hysteria started only in 2022, once OpenAI released ChatGPT - everyone’s new obsession.
Whatever happened next still feels like a fever dream:
OpenAI releasing a voice model that sounds just like Scarlett Johannson.
Microsoft’s AI adding a ‘Guess the cause of death’ poll to an article concerning the death of a young water polo coach in Australia. Readers were asked to choose from three options: murder, accident or suicide.
Google’s Gemini generating factually inaccurate images, some of which were racially insensitive, such as depicting German Nazi soldiers as Black and Asian people.
Yet this exact technology is going to receive over $200 billion in investments by 2025 according to Goldman Sachs (from August 2023).
The same Goldman Sachs (this time from June 2024) released a report saying that the return on investment for AI might be disappointing.
AI technology is exceptionally expensive, and to justify those costs, the technology must be able to solve complex problems, which it isn't designed to do.
The starting point for costs is also so high that even if costs decline, they would have to do so dramatically to make automating tasks with AI affordable. In our experience, even basic summarization tasks often yield illegible and nonsensical results.
Jim Covello, the head of Global Equity Research at Goldman Sachs
AI might just be the tech world's latest shiny object, and boy, do we love shiny objects.
The Model Collapse
So what’s the issue? Let’s just feed more data into AI and increase GPU capacity.
Well, that’s not that simple. See, many people in the industry seem to believe in some sort of scaling law, i.e. that doubling the amount of data and compute capacity will double the capability of AI models.
If we feed ChatGPT with even more data from Reddit, it might just become even more ruthless and offensive, but will not magically gain cognitive abilities and human emotions. The issue is the quality, not the quantity of data, and high-quality data is not cheap and easily available for AI models.
Additionally, with more and more content being AI-generated, the models will just start learning from themselves. You wouldn’t believe how many scientific publications include the word ‘delve’, the favourite word of ChatGPT.
Generative AI produces outputs that, based on its training data, are most probable. That means events that seem to be less probable, whether because of flaws in an algorithm or a training sample that doesn’t correctly reflect the real world, will not show up as much in the model’s responses. And as each generation of AI misunderstands or forgets underrepresented concepts, it will become overconfident about what it does know.
I recommend you to read the recent study on this phenomenon called The Curse of Recursion: Training on Generated Data Makes Models Forget. The study tested how model collapse would play out in various AI programs - think GPT-2 trained on the outputs of GPT-1, GPT-3 on the outputs of GPT-2, GPT-4 on the outputs of GPT-3, and so on, until the nth generation.
The program at first fluently finished a sentence about English Gothic architecture, but after nine generations of learning from AI-generated data, it responded to the same prompt by spewing gibberish:
architecture. In addition to being home to some of the world’s largest populations of black @-@ tailed jackrabbits, white @-@ tailed jackrabbits, blue @-@ tailed jackrabbits, red @-@ tailed jackrabbits, yellow @-.
You are what you eat.
The Bubble This Time
NYU prof Aswath Damodaran has calculated that Nvidia, to justify its valuation, will need not only to continue to dominate the market for AI chips, forecasted to grow aggressively, but also to dominate another market of similar scale. Think about this: Built into the price of Nvidia is the expectation that the company will find another market as big as AI, and achieve similar dominance.
So the question is, are we in AI Bubble?
Of course we are, have you seen what ChatGPT can do?
The economic potential of AI is enormous, there are over 70K AI startups worldwide, and the social media buzz does not seem to stop any time soon. And that is what makes a bubble inevitable. The bursting of today’s AI bubble may not be as problematic as the bursting of the dot-com bubble as many companies spending money today are better capitalised than those spending money back then. But remember, the bigger the hype, the louder the pop.
The AI hype bubble is deflating, and it will have a very significant impact on the future of this technology:
A sharp decline in investment in AI startups. Venture capital firms and other investors would become more cautious, resulting in a slowdown of AI research and development.
A focus shift from a dream to reality. Companies will turn from speculative and experimental AI projects to products and services with proven market demand and revenue potential. This could finally lead to a more practical and application-driven phase in AI development.
A wake-up call for the society. Us, ordinary folks, will develop a more sceptical and cautious attitude toward AI. There might be more room for thoughtful discussions about the ethical implications and issues such as data privacy, algorithmic bias, and the impact of AI on employment.
Over time, investors and the public will develop a more balanced and informed perspective on AI's capabilities and limitations, as the industry stabilizes, and the confidence in the technology gradually returns. The world will eventually heal from the AI craze, but the burst will take a few dozen companies with it.
Eventually, AI may revolutionize everything. But first, it needs to survive its midlife crisis.