The challenge of improving WordPress’s security reputation is perennial – and this week, a new report on WordPress software vulnerabilities made matters much more confusing. The report may be a great lead magnet for Patchstack, a security company, but its sloppy data created an unnecessary setback for the WordPress community as a whole.
In today’s post, I’ll dig into what I think are the flaws of the new report, the areas where I think it highlights important future improvement opportunities, and my thoughts on how we can help improve WordPress’s security reputation.
Sloppy statistics
The lead statistic in Patchstack’s report was a huge fail – and unfortunately, it was amplified when WP Tavern repeated it: “WordPress Ecosystem Records 150% Increase in Security Vulnerabilities in 2021.” While I appreciate Patchstack’s hard work and WP Tavern’s journalism, this is sloppy data reporting that makes the whole community look bad for no reason. A charitable interpretation is that everyone is doing their best with the available data; a less positive one is that Patchstack benefits from the appearance of increased security risks because they sell software that mitigates those risks.
Let’s dig into this 150% number and see where it came from. Patchstack says:
“In 2021, Patchstack added nearly 1500 new vulnerabilities to the Patchstack database. These vulnerabilities were in WordPress plugins, themes, and WordPress core. [Rob’s note: only one vulnerability of 1,500 was in core, and it was because of a third-party dependency.]
If you compare these numbers with 2020 where we saw nearly 600 new vulnerabilities, it’s clear that 2021 has been an exceptional year for the security of the WordPress ecosystem.” [Rob’s note: This is not, in fact, clear.]
The problem here is that they appear to be counting every vulnerability as equal, when in fact that does not reflect the reality of WordPress security. Some open questions:
- How many installs do the vulnerable plugins have? At the very least, this should be our denominator – that is, “we saw X vulnerabilities per Y installs.” That would give us a more realistic picture of how meaningful these vulnerabilities are.
- How serious were these vulnerabilities, and for how long were users exposed before an update was published? A useful statistic would be: “we saw X days of exposure to Y serious vulnerabilities affecting Z installs.” The result of this would likely be a very small percentage of sites being exposed for a very small period of time. In fact, Patchstack’s own report says that just 35 of the 1,500 vulnerabilities they found were “critical,” and only two critical bugs were found in plugins with more than 1 million installs each. None of this adds up to a “150% increase” in actual risk.
- Has the total number of plugins on the market increased? Presumably yes, since earlier in the report they mention the growth of WordPress in terms of its market share. This would naturally lead to an increase in total vulnerability reports even if the per-install rate is unchanged.
- Did Patchstack change their reporting methodologies? Are they just doing a better job keeping up with the database versus last year? Presumably they are growing and getting better at tracking this data over time, so there’s a possibility that we are just seeing an improvement in reporting of “medium”-level vulnerabilities (which make up 76% of their 1,500 vulnerabilities) rather than an actual increase in the prevalence of security bugs.
The 150% number is a striking headline but is in fact meaningless. It does a disservice to the whole community, particularly the many developers who struggle with non-technical stakeholders who “heard that WordPress is insecure.” Where might they have heard this? From people releasing sloppy statistics.
The forgotten story: Core is way more secure than last year
Much of the Patchstack report (including that suspect headline number) is based on a comparison to the 2020 report. Patchstack chose to highlight the “growth” in the raw number of plugin vulnerability reports, even though there are some unanswered questions that make that assertion dubious. What they didn’t mention at all in the 2021 report is that core WordPress vulnerabilities went from 22 in 2020 to just one in 2021.
I suggest a replacement headline: “WordPress core got 95% more secure!”
Too much information
WordPress admirably releases a ton of information on its security issues – and third parties like Patchstack, Wordfence and many others do a great job of logging and alerting the community to these issues as well. Unfortunately, this transparency also puts us at a bit of a public-relations disadvantage.
How many security vulnerabilities did Shopify and Squarespace have last year? No one knows!
This is the double-edged sword of open-source transparency. WordPress has a huge market share, and it does a great job of keeping the world up to date on its security, and so the few security issues that do happen get a disproportionate amount of attention (example headline from WIRED: “Millions of WordPress sites got a forced update to fix a serious bug“).
Obviously, we should keep being transparent about security issues. That said, it would benefit everyone if we could figure out a better public-relations solution to these announcements. When writers use only raw numbers (millions!) or low-context statistics (150%!), it degrades the WordPress brand for no good reason. There are 455,000,000 WordPress sites, which makes the fact that 3 million run Updraft (the plugin from that WIRED headline) less alarming.
Possible alternate headline: “0.65% of WordPress sites get an important update.”
The bug from the WIRED article did not even affect all sites with Updraft installed, since it was only an exploit if you had non-admin members signing up for your site. (If they were aware of the bug, they could download a backup of your site without authorization.) Even with the exploit in place, a malicious person couldn’t actually change your site in any way. In other words, this exploit only exists if you have non-admin members and only matters if you store non-encrypted sensitive information on your site. That means even the “3 million installs” number is not really correct – only a fraction of those sites were ever at risk.
Most importantly, this issue was reported responsibly and fixed within less than 48 hours. To me, this seems like a success story – it certainly doesn’t deserve a shocking headline in WIRED Magazine.
A better headline for next time: “Overnight success: High-profile WordPress developer instantly secures the 0.3% of sites affected by bug.”
While we can’t stop sensationalist journalism and flashy headlines entirely, we should be making a concerted effort to frame security fixes appropriately. A security release means good people are building good software – not the other way around.
Taking out the trash
I’ll end with a point where I agree with Patchstack: the WordPress plugin directory would benefit from better garbage cleanup. Patchstack cites seven plugins that were removed from the WordPress.org directory without ever getting a security patch – which means anyone who had these installed now has an orphaned, insecure plugin on their site and will never get a notification about the problem. While these plugins seem pretty obscure and likely have low install rates, this is still not a great approach. At the very least, in the future, the WordPress Security team should push a security release themselves if a plugin has a known exploit only to get that update notification onto end-users’ dashboards.
Fortunately, these issues are rare – Patchstack says that just seven out of the 50,000+ plugins in the directory were removed because the developers abandoned them without fixing a known critical security bug. Since those plugins had low uptake anyway, they affect an infinitesimally small percentage of actual WordPress sites. Of course, you wouldn’t know this from the Patchstack report – here’s their irresponsible framing: “A whopping 29% of the WordPress plugins with critical vulnerabilities reported in 2021, received no patch from their developers.”
Needless to say, we need WordPress security messaging that doesn’t come from people who sell WordPress security software. Many major WordPress companies have done a great job with this on the SEO front (for example, a search for “Is WordPress Secure?” responds with a resounding “Yes”), but Google News is still a wasteland of poorly framed articles about security patches. My hope is we can all put in a more concerted effort to cite realistic security statistics, rather than parroting sensationalist press releases as if they’re indisputably true.
