So this week gave us maybe the most 2023 headline possible: a single wrong fact in a company's own promotional GIF knocked something like $100 billion off its market value in one day. I've been chewing on this since yesterday and I still think it's the most interesting tech story going right now, more interesting honestly than the actual chatbot it happened to.
Quick recap for anyone who was offline: Google announced Bard, its answer to ChatGPT, on Monday. Fine, expected, everyone saw that coming since December. But then Reuters went and actually read the promo material Google put out to show off Bard, and found that one of the example answers claimed the James Webb Space Telescope took "the very first pictures of a planet outside of our own solar system." Which is just wrong. That happened back in 2004, done by the European Southern Observatory's Very Large Telescope, nearly two decades before JWST existed. Not a subtle error, not a matter of interpretation. A search-answer engine got a search-answerable fact wrong, in the ad for the search-answer engine.
And then Wednesday, Alphabet stock dropped about 7-8% and wiped out roughly $100 billion in market cap in a single trading day. For one factual error in a marketing GIF. I keep sitting with that number because it feels almost disproportionate to the actual mistake, and also completely proportionate to how nervous the market is about this whole category right now.
Here's the part that actually interests me though, not the stock move, the timing of it. Google has been sitting on this kind of language model tech internally for ages — LaMDA's been around, Google engineers have been talking about this stuff in papers since before most of us had heard of ChatGPT. But they moved cautiously, reportedly because of exactly this kind of risk: a model that sounds confident and is sometimes just wrong. Then OpenAI ships ChatGPT in November, everyone loses their minds over it for two months straight, and suddenly Google's rushing a Bard announcement out the door to avoid looking flat-footed. And the very first public artifact from that rushed announcement is a factual error that a decent editor should've caught in about four seconds of googling (the irony of "should have googled it" is not lost on me here).
I don't think this makes Bard bad, to be clear. I think it makes the whole current moment kind of unstable. These things generate plausible-sounding sentences, and plausible is not the same as true, and apparently even the companies building them are shipping the errors into their own launch materials because nobody double-checked. If the people writing the ad copy for the AI didn't fact-check the AI, why would I trust it to fact-check anything for me at 11pm when I'm too lazy to open a second tab?
Small tangent, but it's my blog so I get one: I have absolutely posted wrong information on this site before, more than once probably, over eleven-plus years of doing this. Nobody vaporized $100 billion because of it. That's the actual gap between "a guy with a blog" and "a trillion-dollar company announcing its answer to the biggest tech story since the iPhone" — the margin for error is not the same, and it turns out the market will make that very clear to you within about 36 hours.
Microsoft, for what it's worth, had its own big AI-and-search event the day before all this, showing off a new version of Bing with a chatbot built in. I'm going to leave that one alone for now because I think it needs its own post once people have actually gotten hands-on time with it rather than just watched a stage demo, and because half the internet is already writing that post today anyway. But put the two stories next to each other and you get a pretty clear picture of the next year of search: everyone racing to ship a chat box on top of their search engine, and everyone finding out in real time exactly how expensive a confidently wrong sentence can be.
Anyway. Check your GIFs before you ship them, I guess. Especially if you're worth over a trillion dollars.