Toggle cloudflare security level in case of high load

I’ve recently had some troubles with some stubborn spiders ignoring the robots.txt settings, spidering the site at high velocity and slowing the server down.

In a recent post, I’ve talked about how to limit the nginx requests per client, but that might be not enough in case of a distributed attack with many IPs. or simple high load preventing users from displaying the site. In case that happens, one action that could be taken is switching the Cloudflare (use if you are not) security settings to “under attack“: Cloudflare will display a page instead of the site, that verifies the user for being real ones, and then redirect the user to the real site.

I’ve created a simple bash script to toggle the site security level automatically using the cloudflare API when the server is under high load. Click on the gist name and read the first comment with the instruction to install it.

The way I use it is in a cron is launching the script every 5 minutes, and set the site to “under attack” when the server load is over 7. I’m pasting the ansible cron template here. Replace the variable with the cloudflare user/email, apiKey and zone id (different for each site). Of course, keep it on a single line.

*/5 *   *   *   *   root    /usr/local/bin/cloudflareSecurityLevel 
{{ cloudflare.user }} {{ cloudflare.apiKey }} {{ cloudflare.zoneId }} 
under_attack ">7"

You can also add another line to remove the under attack mode, e.g. when the load is under one

19 *   *   *   *   root    /usr/local/bin/cloudflareSecurityLevel 
{{ cloudflare.user }} {{ cloudflare.apiKey }} {{ cloudflare.zoneId }} 
medium "<1" > /dev/null 2>&1

How to limit nginx requests per client

Nginx has an interesting and powerful module, ngx_http_limit_req_module

This module allows limiting the number of requests per client (E.g: max 1 second per request).

To use it, define the zones (rules) just once at the nginx level (E.g. place into `/etc/nginx/conf.d/zones.conf`). See the following example

limit_req_zone $binary_remote_addr zone=myZone:10m rate=30r/m;

This rule defines a zone called “myZone” that limits clients to max 30 requests a minute (1 request every 2 seconds) per client. 10m is the amount of memory that can be used for nginx to remember. More clients require more memory of course.

To use this rule, place the following inside a “location” directive in the nginx site config

location ~ ^/index\.php(/|$) {
# …
limit_req zone=myZone burst=10 nodelay;

This adds the rule for the specific location matching. The burst=30 setting allow 30 consecutive requests, but – if all of them are performed immediately – then it’ll take another 20 seconds before the client can perform another request.

An interesting article about it here

PHPStorm cache on virtual RAM disk [MacOS]

When using a SSD disk, and enough RAM on the system, I always prefer to move software cache files into a virtual RAM disk, in order to extend the disk life, and also gain more speed.

The command on MacOs to create a 512Mb Virtual volume (under /Volumes/RAMDisk) is

diskutil erasevolume HFS+ 'RAMDisk' `hdiutil attach -nomount ram://524288`;

After that, you can create your symlink, and move that command in a startup script

PHPStorm script

Here is the script I’m using to create the virtual Disk and the index and cache directories for PHPStorm. Of course you need to symlink them the first time you use them (see the comments in the gist).

You’ll lose the cache and index next time you restart your computer or unmount your RAMDisk, but that’s something I actually prefer, to keep the cache clean from old projects and libraries.