WordPress Cache Plugins Are Not Enough
Reverse Proxy and Object Caching – Performance at Any Scale
WordPress cache plugins can help your site go faster – improving page load times and your ability to serve much more traffic without degradation in performance. But there will come a time when a cache plugin will not be enough.
All it takes is a huge peak of traffic. If content on your site makes it big and you get sudden attention on social media or other channels – the concentrated burst of traffic can bring your site to its knees. A huge success can turn into a performance disaster.
This page will answer the following questions:
- Why are WP cache plugins not enough for large scale?
- How do the pros handle WordPress caching?
- What additional tools do they have beyond the regular cache plugins?
- What benefits can you get from these additional tools?
- How can you get super-scale caching built into WordPress?
Note: We’re the makers of Pantheon, the high-performance, enterprise-grade hosting platform for WordPress. The info below is based on our experience hosting tens of thousands of demanding WordPress sites and helping them scale up.
Why Cache Plugins are Not Enough – Meet the Reverse Proxy
What’s the fastest WordPress cache plugin on the planet? Probably Batcache, a blazing-fast plugin used to speed the world’s biggest WordPress server, WordPress.com.
Now see this chart from Joe Hoyle comparing Batcache with a different option, Varnish. The green graph shows what happens to even this industrial-strength plugin as you increase the number of concurrent users on your site:
As the load increases to 20, 50, 100 and more concurrent users, response times start climbing to 10, 25, 40 and even 70 seconds. It only happens at the higher levels of traffic, but that’s where we want to be, right?
This applies to many popular WordPress cache plugins – such as WP Rocket and W3 Total Cache – which serve web pages using WordPress and its underlying PHP engine. They rely on compression, CSS minifying and other techniques to reduce page load time, and this works at the small scale, but suffers when WordPress itself starts to choke on big traffic.
What is the orange graph that remains stable even when the number of concurrent users goes through the roof? That’s a reverse proxy.
A reverse proxy (Varnish in this case) is an intermediary service that runs independently, not as part of WordPress, and accepts requests to the WordPress server. Like a cache plugin, it provides a pre-saved copy of a WordPress response for a limited time.
The difference is that while cache plugins are driven by the WordPress server, which can get tired as load increases, a tool like Varnish is specialized at serving cache requests and can be literally up to 100x more efficient than a PHP web server.
That generates the amazing flat graph as traffic increases to mega-site levels.
If you want to get started with Varnish, this post provided by the makers of Varnish explains how to setup Varnish on WordPress.
Static-File-Based Plugins vs. Varnish
Some WordPress cache plugins work differently – they save static files of your dynamic WordPress pages and reroute web visitors to those static versions.
However, there are a number of problems we’ve seen with cache plugins based on static files:
File system trouble – One of the downsides of file based caches like WP Super Cache is that on certain file systems (especially a networked file system like NFS) the disk can significantly hurt performance when you write/delete/change many files at once. This can cause your site to either not cache data (making it slow) or serve badly cached data (making it look broken). Varnish keeps cached page data in RAM, which avoids this problem.
Plugin incompatibility – Due to the number of different ways that data can be rendered/assembled using WordPress, not all plugins can take advantage of different caching solutions and run into compatibility problems. With Varnish, caching is done on the edge and only individual elements are cached. This means you remain fully compatible with plugins and dynamic content.
Web server configuration required– A number of plugins, like WP Rocket, require specific settings on the web server to get the cache to work. In many cases, the webmaster can’t change web server settings or has a non-Apache web server like NGINX.
Inefficient compression handling – Some WordPress cache plugins compress most of your files using GZIP every time they’re requested. Varnish caches the compressed pages and delivers them as-is for web browsers that support GZIP (99%), and uncompresses the files on the fly, which is less efficient, only for browsers that don’t support GZIP. The same approach is taken by CDNs. This means Varnish’s compression handling is better optimized for the vast majority of users.
No failure isolation – Varnish and CDNs can recognize when something is wrong on your server, and help you bridge over this time by delivering slightly staler (but still functional) content. This can significantly reduce the impact of downtime. Most WP page caching plugins do not do this, making web server downtime full downtime.
Disclaimer: We aren’t affiliated with Varnish, but at Pantheon, we feel strongly that it’s the best option for large-scale WordPress. That’s why we’ve implemented it as part of our WordPress hosting platform.