Lessons learned from the Log4j vulnerability

Beyond SQL injection

The problem is that Log4j 2 gave us a new vector for code injection, via JNDI. Bad actors can get your virtual machines to download and execute arbitrary code through a simple JNDI reference embedded in text. It’s SQL injection, part two.

Like SQL injection, it can be mitigated if you properly sanitize logging output, and simply don’t use user-supplied content as a logging template. Most static code analysis tools, such as FindBugs or PMD, check for such things which is why it’s a best practice to include such tools in your continuous integration builds. However, also like SQL injection, mitigation efforts rely on your entire stack to do the right thing, and you can’t rely on that assumption.

There are things you can do to mitigate exposure to risks like the Log4j 2 flaw. The most important step is to update your Log4j 2 dependencies when you install Java.

If you use Log4j 2 anywhere in your Java applications, update to the most recent version of Log4j 2. According to the project’s site, if you’re on Java 6 update to 2.3.2. If you’re on Java 7, update to 2.12.4. If you’ve the version of Java you installed is more recent, update to 2.17.1 or later.

The Log4j 2 blame game

The next thing to think about is, how did this happen? The Log4j 2 security hole isn’t really a hole — it’s a cavernous tunnel. It’s hard to understate the potential severity of the problem. How did such a simple and common dependency as logging create such a mess?

The Swiss Government Computer Emergency Response Team provides this chart to describe how a Log4j 2 LDAP injection attack is executes on the server.

The short answer is offered without judgement: pride. There’s a line in Michael Crichton’s Jurassic Park, where Ian Malcolm scolded the people who created a Frankenstein’s Monster: “Your scientists were so preoccupied with whether or not they could, they didn’t stop to think if they should.”

It’s an apt quote for the Log4j 2 havoc. The cause of the JNDI attack vector is the ability to look up logging content by providing an embedded JNDI reference, one that can access an external site. That JNDI content can be a serialized Java class, which can run arbitrary code in the virtual machine. It sounds like a really neat feature, if you should need it.

But should you ever need it? How often does one actually want to look up a logging template in an external resource? I think it’s fair to assume someone wanted it somewhere at some point, because they wrote the code for it. But is it actually a feature the core library needs?

Writing that code requires some degree of hubris, although I’m sure the original author justified it in some form or fashion. (I certainly hope so; if someone provided this feature on a lark with no requirements, this is a tragedy of Homeric proportions.)

Log4j 2 vulnerability’s origins: Feature creep

The true hubris, observed without blame, belongs to the Log4j community which approved the feature to include in the library. any actual failure occurred in the process, it was here. Perhaps a programmer needing the JNDI lookup feature for some reason, but that merge request went to the Log4j community for inclusion. They authorized it.

I say this with no contempt or ill will toward the Log4j community. There but for the grace of God go I — surely I’ve approved features that made no sense to me at the time, and saw no harm in them until later analysis.

Yet this is how the problem began: unchecked feature creep, pride, and inattention. Instead of saying, “This sounds like a neat feature, but it isn’t part of Log4j’s mission,” people said “sure, why not, what’s the harm?” – and now we see the answer to that question.

The community impact of this bug is still more emotional than anything else. Java’s always had potential security holes but overall  it has a pretty good record. Most holes show up as did the Log4j 2 bug — in libraries that didn’t anticipate how bad actors could use specific features, and ultimately repaired fairly quickly.

That doesn’t erase the black mark on Java’s record. We scoffed and sneered at JavaScript’s ecosystem suffering when left-pad’s removal broke everything – and now we have our own variant of the same problem.

Lessons learned from Log4j J2

But what’s the real long-term impact? It depends on us, and what we can learn from all this.

We need a manageable update process for dependencies — and we now see why.

We must better understand how the communities that spring up around libraries we depend on work, and be more vigilant within those communities. We remind ourselves that libraries have a specific function, and features that don’t directly contribute to that function should be integrated carefully, if at all.

We can weather this storm around Log4j 2; we’ve handled worse. We just need to approach this situation rationally, as we should any situation.