[Edit] The below post was redacted in the heat, right after the initial publication. Should you wish a summarized, and arguably more neutral tone, view of the situation, please consider reading the follow up.
A
recent post on a security blog has made a wonderful job at sexing up a subtle bug affecting
LZ4, claiming that it would result in remote code execution. Given the current spread of use of LZ4, from every modern
Linux distro, including critically
Android, to modern file systems such as
ZFS, shipped with
FreeBSD and
Illumos, through widely deployed databases such as
MySQL, or big data nodes powered by
Hadoop, the implication is that it must be a pretty big deal.
The detailed article then makes a minimal job at describing the strong conditions required to trigger that risk, but these explanations are lost, scattered within the content of an overly long and boring technical article, ensuring most readers will stop at the dramatic headline. The present article has the objective to counterweight those "high stakes" claims.
I won't spend too much time underlining the vast difference between reporting a bug for correction to the relevant team, and creating a dramatic communication for maximum coverage without even bothering if a fix is available for the exploit. The first behavior is valuable, helpful and welcomed. The second one is at best irresponsible, if not downright harmful. A security auditor is expected to know this simple common sense, and behave accordingly.
But perhaps more importantly, this bug wasn't unknown, which contradicts the statement that it was found through careful code study. The risk was identified and
openly published on the LZ4 public issue board by
Ludwig Strigeus, the creator of
µTorrent. Instead of trying to make a headline, he even proposed a solution for it (unfortunately with some side effects). Ludwig made a fine job at describing the risk and opening a discussion.The bug was classified minor (rightly or wrongly) for reasons which will be detailed below. Thus, it was tracked for correction, but with no urgency, trying to design a fix without the initial side effects. After multiple partial improvements, a change of code related to the
new streaming mode finally allowed such a fix to be created, closing the issue (by chance, merely a few days before DonB second "disclosure").
Let's now get into the technical details.
In order to use the vulnerability, a number of conditions must be met.
A first minor one is : it is necessary to work within a 32-bits environment. 64-bits is totally unaffected. That basically put most server-side applications outside of the risk zone.
Second, the attacker need to forge a special compressed block to overflow 32-bits address space. This is possible ... if the compressed block is something like 16 MB.
There is just a little problem with that : the legacy LZ4 file format is limited to 8 MB blocks. Any value larger than that just stops the decoding process. 8MB is not enough to trigger a problem, since 32-bits application are restricted to low address ranges. The newer streaming format is even stricter, with a hard limit at 4 MB. As a consequence, it's not possible to exploit that vulnerability using the documented LZ4 file/streaming format.
Well, but what about programs which use their own, undocumented, format ?
Indeed, same condition applies. To be exposed to this risk, very large blocks of 16MB or larger must be read by the decoder.
Does that happen ?
Let's have a look at several high-profile programs using LZ4.
ZFS ? Max block size is 128 KB.
Lucene ? Typical index size is 16 KB. It could be a bit more, say 64 KB, that's still far short of the objective.
zram maybe ? nope, 4 KB memory segments.
Linux kernel ? The boot section has to decompress a multi-megabytes kernel into memory, surely it can match the limit ? Yes, it does, but it uses the LZ4 Legacy File format, which limits each block to 8 MB maximum. OK, maybe AAA games such as
Guild Wars 2 ? nope, real-time protocol is the realm of short messages, the protocol itself doesn't allow anything beyond a few KB. And so on, and on.
At the end of the day, none of the known implementation of LZ4 is exposed to this risk.
Sure, someone creating some private program at home could get into this pitfall, but this situation is not exactly a "security risk". Only widely deployed programs actually matter, since they are the potential targets by hackers or surveillance organizations.
Basically, most user programs employ LZ4 for small data packet structure, way below the critical limit. Programs which generate and distribute large compressed blocks (notably the
lz4c pos-x compression utility, distributed within Linux Distro) use the
documented framing format, which limits block size to 4 or 8 MB. Remove also from the list programs which never take "externally provided" data as input, they can't be targeted either.
So sorry, this is not a "new heartbleed" situation the author seems to dream for.
Sure, since
a fix is currently available, it's nonetheless a good move to close this risk. But I wouldn't flag that action as "critical", since most applications stand outside of the "custom compression format using large blocks of > 8 MB on 32-bits system, and receiving data from untrusted external sources" scenario.
It's one thing to tell there is a potential vulnerability that should be fixed, to ensure it does not become exploitable in the future. It's a totally different thing to pretend there is a dangerous RCE (Remote Code Execution) exploit currently active on the Internet, which is scary. DonB article cleverly conflates the two, implying the second to create a flashy headline, while disseminating some minor disclaimer elements throughout the long article to pretend, whenever necessary, having said the first. That's clever, but conflating gravity level to grab some free ad is not a respectful behavior from a security firm.
The world is already full of real security threats, from which
heartbleed is just one of the most recent examples. Triggering too many alarms to grab a bit of fame is a good way to weaken the power of
future alarms. You can guess that when every headline claim a "new heartbleed" situation, no one will pay attention to the real next one which
will matter. And
that's dangerous.
The long list of "credits" at the end of the article is another reason for caution. It happens I know some the influential names listed there, and they told me they barely heard indirectly about the guy. So why are they listed there ? Fame by association ? Sure, please thank
Linus Torvald for "coordinating and advising" on the issue. This is becoming plain ridiculous.
Anyway, should you feel nonetheless at risk now, please don't hesitate to
update your LZ4 version. It's a good thing to do anyway, and as stated previously, the vulnerability was already patched.
