Cloud Server: From 73 To 26127 Hits With A Properly Configured Caching Layer


I was recently involved in this reddit thread discussing memory utilization and server performance.

A user was complaining about his Cloudways server, deployed over Digital Ocean, having not much free memory. He was comparing it with a server launched directly over Digital Ocean (same size) and configured using Serverpilot tool that had much more free memory. 

There is the misconception that a server with lots of free memory will perform better than a server with less free memory. While it is certainly true that you need free memory to run your system (and the OS is perfectly able to handle this), what is key is to understand which is the configuration of the server and how the memory is used, cause this will determine performance in the end. 

The key points in the thread:

  • Memory usage really depends on the type of website and plugins you are running.  (blazedd)
  • It all comes down to the server configuration. It really depends on how efficient you want to be and how much effort you want to put in. (Ir0nMann)
  • Optimized for performance, not resource usage. Engineering is all about comprises. (LostDatagram)
  • Adding varnish and memcached will soak up quite a bit more memory to provide a lot better performance. (ivosaurus)

And this is really the point. As I explain there:

As few of the people here have suggested, it all boils down to server configuration. Specifically our stack is:
nignx (static content) -> varnish (caching dynamic content) -> apache (serving dynamic content not cached) -> memcached (caching database queries) -> mysql
Now I don't know the specifics on how serverpilot configures servers, but as you can see we have a heavy caching layer to ensure, as very pointedly LostDatagram mentioned, performance.
Surely caching uses memory and because of this you can see low free memory in a 512MB server. Now, there is nothing specifically wrong with it as long as there are no other memory hungry WP plugins or similar running on the server. Even in this case, we deploy on all our servers a 512MB swap configured to only kick in to avoid processes being killed due to lack of memory, so adding an additional protection layer for the server while keeping performance.

After challenging people to test this hypothesis I couldn't resist just doing it myself and, not surprisingly, the results where quite different. 


Test Specifications

Digital Ocean server


Same default wordpress configured on both servers

Load test using

Up to 3000 concurrent users over the span on 180 seconds (huge surge)


Cloudways + Digital Ocean

So 26127 successful hits with 12281 errors & 3686 timeouts

As you can see the server suffered due to the high load, but the strong caching layer allowed it to survive and even stabilize performance after a while. 


Serverpilot + Digital Ocean

71 successful hits and at around 250 concurrent users the server couldn't handle load anymore. Load time was already 10s at this point.


And the only difference here, is an strong and properly configured caching layer. So take away: