So much for a lazy Saturday. I had grand plans today involving coffee, the couch, and absolutely no terminal windows, and instead I spent about four hours patching bash on every box I have a login for. If you run anything on a Linux server and you haven't done this yet, stop reading this post and go do it. I'll still be here.
By now you've probably heard about Shellshock, the bash vulnerability that got a public writeup this week (CVE-2014-6271, if you want to look it up). The short version: bash has had a bug for something like two decades where it'll execute code stuffed into an environment variable if that variable is formatted just right. That doesn't sound like much until you remember bash is the thing quietly running behind CGI scripts, DHCP clients, SSH forced commands, and a thousand other places most of us never think about. Somebody described it as "worse than Heartbleed" and honestly, after spending my afternoon on this, I don't think that's hype.
I've got three boxes I actually care about: the droplet this blog lives on, a little box I use for a side project that barely gets traffic, and an ancient VPS I keep around mostly out of guilt because it still has some PHP scripts on it from 2012 that I'm scared to touch. Two out of three were running Ubuntu versions old enough that apt-get update took actual minutes just to refresh the package lists, which tells you something about how often I log into them.
The test everyone was passing around is dead simple, you just run this in a terminal:
env x='() { :;}; echo vulnerable' bash -c "echo this is a test"
If it prints "vulnerable" before "this is a test," congratulations, you've got the bug. All three of my machines did. That's a little embarrassing to admit on a blog that's ostensibly about tech, but I'd bet most people reading this would get the same result if they checked right now.
The actual patching was mostly painless, apt-get install --only-upgrade bash and a reboot, but the process of remembering which of my domains still points at which server, and which of those servers has cron jobs I forgot I set up, took way longer than the fix itself. I found a cron job on the old VPS that's been silently failing since some time last year because it references a database that doesn't exist anymore. Nobody noticed because nobody was looking. That's kind of the whole story of Shellshock in miniature, honestly, a huge pile of infrastructure just sitting there running, unpatched and unwatched, until someone points a flashlight at it.
The other thing that made this week annoying was Apple pushing out iOS 8.0.1 on Wednesday and then yanking it an hour later because it broke cellular service and Touch ID on the new phones. I don't have a 6 or a 6 Plus so I dodged that one, but between that and Shellshock it's been a rough week to be the person your family calls when their phone stops working. My mom texted me Wednesday night asking why her signal bars vanished and I had genuinely no idea until I read about it the next morning. Apple got a fixed version out within a day or two, which is fast for them, but it's a bad look right after a launch that big.
What bugs me most about the bash situation isn't the bug itself, bugs happen, it's how long it sat there. Two decades is a long time for something this exploitable to just exist in code that ships on basically every Linux and Mac machine on earth. I don't have a tidy conclusion about what that means for how we build software, other than that I'm going to be a lot less smug about my little home server setup from now on. I used to think of patching as something you do when you get around to it. Not anymore, at least not this month.
Anyway, if you skipped the intro and jumped straight here: go run that test command. It takes ten seconds and it might save you a much worse Saturday than mine.