There is a new zero-day security vulnerability known as Log4Shell that hit the world last week. Here’s what you need to know.
Some Background
Log4j is a wildly popular Java logging utility maintained by the Apache Foundation. On the 12th of July in 2014 Apache released Log4j v2.0 which, among other things, added a plugin for an existing Java feature called JNDI. This plugin allows Log4j to access arbitrary external resources (like a website) and execute commands. The feature ostensibly was to support logging configuration and convenience but would turn out to have many unintended consequences.
Seven and a half years later, just before 7:30 am Pacific Time, security researcher Tang Xiaofeng committed a small file that read, “Apache Log4j 远程代码执行,” or in English: Apache Log4j Remote Code Execution.
Over the next seven minutes, the researcher added more files which demonstrated a working attack using the vulnerability in Log4j. Alongside the example code, the researcher added a recommendation to disable the feature which opened up the vulnerability before calling it a day, tweeting out the code on their P0rZ9 Twitter account.
An official CVE was published by NIST less than 24 hours later. That same day Lunasec christened it Log4Shell.
We now know that back on the 24th of November, the Alibaba Cloud Security Team had privately contacted Apache about the vulnerability, and a partial fix for what would become Log4Shell was already in the pipeline by the 5th of December. This fix was released the next day with Log4j v2.15. This fix, however, became associated with a fresh denial-of-service attack vector, as well as still leaving the door open for similar exploits. A second CVE was published by NIST on the 14th. With Log4j v2.16, the JNDI functionality was finally fully disabled by default. But even though the patch exists, how long will it take before all the vulnerable systems are updated?
What is Log4Shell?
Log4Shell is a remote code execution vulnerability, or RCE, that is embarrassingly easy to pull off. A successful execution allows an attacker to have a computer do nearly anything they could want. Reviewing our server logs for attempts reveals that most people currently trying the exploit are asking the server to download a remote payload. That payload either executes some nastiness or are messages from well-meaning people trying to tell the server administrator about the problem.
Reporting on this vulnerability has been ramping up over the last few days. , Unfortunately, however, a lot of the essential technical information on it is squirreled away in Twitter threads and other formats that search engines don’t index well.
It is difficult to overemphasize how significant this vulnerability is. Millions of programs and potentially billions of devices out there are currently unpatched – and may never receive a patch. This exploit is so trivial to pull off that it is already being incorporated into delivery packages for scanning, cryptojacking, ransomware, botnets, outright data theft, and so much more.
There is now evidence that Log4Shell was being used in the wild even before its December 9th public disclosure. The attacks are continuous and ongoing despite a fix being available. These will fund the next wave of development of malware and phishing campaigns.
Log4j at Valimail
Emails continue to come in from vendors saying they aren’t vulnerable to Log4J because we don’t personally use Java, and it would be easy for us to do the same. Valimail is primarily a Ruby and Go shop and we don’t use Java anywhere in our codebase. However, that doesn’t entirely answer the question of vulnerability, as many tools, devices, and services used by the vast majority of the planet use Java and the ubiquitous Log4j framework.
Does your organization use the ELK stack? Does it use Salesforce? Snowflake? On-premise Atlassian? Do you have any IoT or smart home appliances? What about your phones? Do you run a Minecraft server on a personal device with work information on it? These are all potential bastions of vulnerable Log4j versions in your footprint that can place an organization at risk. As a security company, we take this risk seriously.
Upon disclosure of the vulnerability on the 9th, we began evaluating our infrastructure for our own attack surface. Since we don’t do Java development at Valimail our approach turned to self-hosted third-party tools that may be vulnerable. We had already been keeping up to date with the latest releases, and thus had fixes in place for most tools before we even knew there was a problem, and in addition, identified a few potentially vulnerable third-party systems for which we deployed updated versions while also taking further steps to mitigate access potential. Using our rapid deployment Kubernetes automation we were able to re-deploy the appliances with no downtime or customer impact.
To verify our security we employed the attack against our own systems during the December 10-13th weekend and found no holes.
None of our systems has been compromised and all of our user data continues to be secure.
We continue to monitor our environment in case of any newly discovered CVEs and as always we do regular 3rd party security audits.
It’s bad out there. But we’re covered.
If you’re looking for a place where you can do great work with great people, check us out. Valimail is the place for you!