Toggle cloudflare security level in case of high load

I’ve recently had some troubles with some stubborn spiders ignoring the robots.txt settings, spidering the site at high velocity and slowing the server down.

In a recent post, I’ve talked about how to limit the nginx requests per client, but that might be not enough in case of a distributed attack with many IPs. or simple high load preventing users from displaying the site. In case that happens, one action that could be taken is switching the Cloudflare (use if you are not) security settings to “under attack“: Cloudflare will display a page instead of the site, that verifies the user for being real ones, and then redirect the user to the real site.

I’ve created a simple bash script to toggle the site security level automatically using the cloudflare API when the server is under high load. Click on the gist name and read the first comment with the instruction to install it.

The way I use it is in a cron is launching the script every 5 minutes, and set the site to “under attack” when the server load is over 7. I’m pasting the ansible cron template here. Replace the variable with the cloudflare user/email, apiKey and zone id (different for each site). Of course, keep it on a single line.

*/5 *   *   *   *   root    /usr/local/bin/cloudflareSecurityLevel 
{{ cloudflare.user }} {{ cloudflare.apiKey }} {{ cloudflare.zoneId }} 
under_attack ">7"

You can also add another line to remove the under attack mode, e.g. when the load is under one

19 *   *   *   *   root    /usr/local/bin/cloudflareSecurityLevel 
{{ cloudflare.user }} {{ cloudflare.apiKey }} {{ cloudflare.zoneId }} 
medium "<1" > /dev/null 2>&1

How to limit nginx requests per client

Nginx has an interesting and powerful module, ngx_http_limit_req_module

This module allows limiting the number of requests per client (E.g: max 1 second per request).

To use it, define the zones (rules) just once at the nginx level (E.g. place into `/etc/nginx/conf.d/zones.conf`). See the following example

limit_req_zone $binary_remote_addr zone=myZone:10m rate=30r/m;

This rule defines a zone called “myZone” that limits clients to max 30 requests a minute (1 request every 2 seconds) per client. 10m is the amount of memory that can be used for nginx to remember. More clients require more memory of course.

To use this rule, place the following inside a “location” directive in the nginx site config

location ~ ^/index\.php(/|$) {
# …
limit_req zone=myZone burst=10 nodelay;
}

This adds the rule for the specific location matching. The burst=30 setting allow 30 consecutive requests, but – if all of them are performed immediately – then it’ll take another 20 seconds before the client can perform another request.

An interesting article about it here

301 redirect in nginx

A permanent server redirect is useful when you move domain, or when you want to redirect
users to the WWW-version

Example for nginx, add the following into /etc/nginx/conf.d/redirect.conf

server {
    server_name oldDomain.com;
    return 301 $scheme://www.newDomain.com$request_uri;
}