So this morning I was drinking my coffee and half-reading Twitter like always, and everyone's suddenly talking about a company called Knight Capital. Never heard of them before today. Turns out they're one of the biggest market-making firms on Wall Street, the kind of place that handles a huge chunk of trades for retail brokers like TD Ameritrade, and this morning some piece of trading software they rolled out went completely off the rails right at the opening bell.
The short version, as far as I can piece together from the news wires: they pushed new trading code live, and some old test routine that was supposed to be dead got triggered instead. For about 45 minutes starting at 9:30am, the system just started firing off a flood of erroneous orders into something like 150 different stocks, buying high and selling low over and over, millions of shares, completely unsupervised. By the time somebody pulled the plug, Knight was sitting on something like $440 million in losses. Their own stock dropped something like a third of its value today. The SEC apparently had to step in and cancel trades in six stocks because the prices got so distorted.
I keep coming back to the number 45 minutes. Forty-five minutes for a bug to burn through nearly half a billion dollars. Thats not a slow leak, thats a firehose.
Ive shipped bad code before. Everyone who writes software has. Ive got a story from a couple years back where I meant to set a cron job to run once an hour and instead set it to run every minute, and it quietly hammered a third-party API a few thousand times before anyone noticed the bill (small bill, thankfully, not a $440 million bill, but you get the idea of that stomach-drop feeling). Theres a very specific kind of dread that comes from watching something you built do the wrong thing at scale and not being able to stop it fast enough. I cant imagine what that felt like this morning for whoever was on call at Knight, watching their own system eat the company alive in real time and not being able to find the kill switch.
What gets me is this wasnt some hacker, some outside attack, nothing exotic. It was leftover code. Dead code that should have been deleted and wasnt, sitting there dormant until the wrong flag got flipped and it woke back up. Ive got old commented-out functions in projects I havent touched in a year that I keep meaning to clean out because I'm lazy, and reading about this made me a little more paranoid about that habit than I was yesterday.
Heres my actual opinion on this, and I know its not a popular one among the finance-blog crowd: this isnt really a story about one company having a bad morning. This is what happens when you build an entire market structure around algorithms trading against each other at speeds no human can meaningfully supervise, then act surprised when one of them misfires. High-frequency trading has been marketed for years as making markets more efficient, tighter spreads, all that. Fine, maybe. But efficient markets built on systems that can lose $440 million in 45 minutes with zero human intervention dont sound especially safe to me, they sound like a car with a great top speed and no brakes. Every time one of these firms has a "glitch" (and this isnt even the first one this year, remember the Facebook IPO mess back in May, also partly a Nasdaq systems problem) the response is some patched-over fix and a promise it wont happen again. It'll happen again. Different company, different bug, same basic problem: nobody notices a runaway process fast enough when its running that fast.
Anyway. Knight is apparently trying to figure out if they can even stay in business after eating a loss that size in under an hour, which is its own kind of remarkable given the company existed for years before this morning. I dont have a horse in this race, I dont trade individual stocks, my 401k is just sitting in some boring index fund doing boring index fund things. But as someone who writes code for a living, this one hit different than the usual quarterly-earnings churn. Its a reminder that a bug isnt always an annoyance. Sometimes its a company-ending event, and it can happen faster than the meeting you'd call to figure out what to do about it.
Going to go double check my own crontab now, honestly.