My nephew has asked me three separate times now why I still haven't seen Spider-Man: No Way Home, and I don't have a good answer for a twelve year old that starts with "well, a logging library." But here we are, four days before Christmas Eve, and that's genuinely where my last two weekends went.
If you run anything on a JVM you already know what I'm talking about. CVE-2021-44228 dropped on the 9th, the JNDI lookup thing in Log4j2, and it was bad in that specific way where the proof-of-concept was about four lines long and worked against half the internet. I patched our build server that first weekend feeling pretty good about myself. Then the 14th happened and CVE-2021-45046 showed up because the original fix wasn't actually a full fix, it just narrowed the blast radius. Then last Saturday, the 18th, there's a third one, CVE-2021-45105, a denial-of-service angle this time, and Apache ships 2.17.0 to close that one out too. Three rounds in ten days. I stopped trusting my own "we're good now" emails to my boss after the second one.
The part nobody warns you about with a vulnerability like this isn't the actual patch, it's the archaeology. Finding every jar file on every box, including the ones some contractor dropped in three years ago and nobody remembers, including the shaded dependencies buried four levels deep where Log4j isn't even a direct dependency, it's a dependency of a dependency of a dependency. I wrote a little script that just grep'd every filesystem I have access to for log4j-core jar filenames and honestly the results were depressing. Some vendor appliance I have zero admin rights on, running who-knows-what version, sitting on the network like a little time bomb, and support won't even confirm if they're affected until January.
I'm not going to pretend I have some grand insight about supply chain security that the entire tech press hasn't already said a hundred times this month. Everyone's written the "one library, half the internet" piece already. What I actually want to complain about is smaller than that: it's that this happened in December. Two weeks before Christmas, when half my team is already mentally checked out and the other half is trying to use up PTO before it resets, and instead we're all in a Slack channel at 11pm comparing Maven dependency trees. My manager sent a message Tuesday that just said "no new CVEs please" like that was a thing any of us controlled.
Meanwhile my nephew's parents took him to the opening weekend and apparently it did something like $260 million domestically, which is an absurd number for a pandemic-era opening, packed theaters and everything from what I hear. I keep meaning to go. I have the popcorn-flavored soda from the theater gift shop sitting unopened on my counter because my sister brought me some as a joke gift, "since you clearly weren't coming with us." It's still there. I might drink it while reading incident reports, that feels about right for this month.
Anyway. If you're reading this and you still haven't gone through every artifact repository you own looking for old Log4j versions, do it this week and not next week, because I guarantee you the people who'll actually try to exploit this over the holiday break are counting on IT departments being short-staffed and distracted. That's not paranoia, that's just how this always goes with a vulnerability this loud and this easy to weaponize. I'd rather be annoyed now than answering questions in January about why something got in.
I did finally get confirmation from that one vendor appliance today, by the way. They're on 2.14.1. Of course they are. That's a Monday problem now, not a Thursday-before-Christmas problem, and I've decided I'm allowed to say that out loud without feeling guilty about it.