How to write and enable custom Fail2ban filters

Previously, I've shared how I caught and blocked a brute force attack on Wordpress xmlrpc.php. I'll use this attack as an example to explain how to write a custom filter and use it to automatically block an attacker.

A short introduction

Fail2ban is a software that watches log files and detect patterns in text. When a pattern is detected, it creates a firewall rule to block the attacker's IP address.

Read this article to learn more about how Fail2ban works, I'll focus only on how to write patterns and enable detection of those patterns on Ubuntu Server 14.04¹.

The first thing is to install Fail2ban:

 sudo apt-get update
 sudo apt-get install -y fail2ban

Look for patterns in log files

In my case, a Wordpress brute force attack on the xmlrpc.php file, it can be detected in the Apache's access log file:

 $ sudo cat /var/log/apache2/access.log | grep xmlrpc.php
 191.96.249.54 - [26/Aug/2016:21:59:56 -0400] "POST /xmlrpc.php HTTP/1.0" 403 471 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"

Indicators I use to identify the attack are:

POST request - contains the payload, credentials
sent to /xmlrpc.php - targeted vulnerability
returning a 403 status code - failed attempt to login

I use RegExr to make sure my regular expression is correct:

https://regexr.com

The pattern I come up with is ^\S+ .+POST \/xmlrpc\.php HTTP.+ 403

Then I replace the \S+ part, which stands for "a sequence of any non whitespace character", with the Fail2ban <HOST> token. The Fail2ban's failregex now looks like this:

 ^<HOST> .+POST \/xmlrpc\.php HTTP.+ 403

Different operating systems or configurations might prefix log lines with additional information and requires to add %(__prefix_line)s after ^

Create a new filter that catch the pattern

I create a new xmlrpc-bruteforce.conf file in the /etc/fail2ban/filter.d directory:

 $ sudo nano /etc/fail2ban/filter.d/xmlrpc-bruteforce.conf
 
 # Fail2Ban filter for WordPress bruteforce through xmlrpc.php file
 #
 
 
 [INCLUDES]
 
 before = common.conf
 
 [Definition]
 
 failregex = ^<HOST> .+POST \/xmlrpc\.php HTTP.+ 403
 
 ignoreregex =

Including the common.conf file allows the use of generic replacement patterns such as %(__prefix_line).

Test the filter with Apache's log file

Thankfully, Fail2ban comes with a testing tool that makes testing straightforward:

 $ fail2ban-regex /var/log/apache2/access.log /etc/fail2ban/filter.d/xmlrpc-bruteforce.conf
 
 Running tests
 =============
 
 
 Use failregex file : /etc/fail2ban/filter.d/xmlrpc-bruteforce.conf
 Use log file : /var/log/apache2/access.log
 
 # Results
 
 ...
 Lines: 1720 lines, 0 ignored, 456 matched, 1264 missed

The test successfully shows that the pattern is found in 456 of 1720 lines contained in the access.log file.

To confirm, I run the test on a part of the file only, where I know how many occurences should be found.

Define a jail to apply the filter

Once the filter is created, I add a new jail by creating the /etc/fail2ban/jail.d/xmlrpc-bruteforce.conf file:

 $ sudo nano /etc/fail2ban/jail.d/xmlrpc-bruteforce.conf
 
 [xmlrpc-bruteforce]
 
 enabled = true                         # rule is enable
 filter = xmlrpc-bruteforce             # given this filter
 logpath = /var/log/apache2/access.log  # watch this file
 maxretry = 1                           # and on first match
 banaction = iptables-allports          # apply this ban
 bantime = 604800                       # for 1 week
 protocol = all                         # for any protocol

maxretry defines how many times a given IP address can match the filter before being sent to jail. The above jail is pretty strict given that there's no legitimate use of the file for this Wordpress installation.

Activate the new jail

To enable jails, Fail2ban has to reload:

 $ sudo fail2ban-client reload
 2015-04-18 16:12:08,321 fail2ban.jail   : INFO   Jail 'xmlrpc-bruteforce' started

Fail2ban jail in action

Fail2ban has created a new firewall rule on the first attempt (ESTABLISHED) and I can see dropped traffic (SYN_SENT) with tcptrack:

 $ tcptrack -i venet0 port 80
 ...
 191.96.249.54:55646 169.125.128.112:80  SYN_SENT    4s   0 B/s
 191.96.249.54:41928 169.125.128.112:80  SYN_SENT    8s   0 B/s
 191.96.249.54:46015 169.125.128.112:80  SYN_SENT    5s   0 B/s
 191.96.249.54:59913 169.125.128.112:80  SYN_SENT    30s  0 B/s
 191.96.249.54:60861 169.125.128.112:80  SYN_SENT    23s  0 B/s
 191.96.249.54:54418 169.125.128.112:80  SYN_SENT    5s   0 B/s
 191.96.249.54:36830 169.125.128.112:80  SYN_SENT    18s  0 B/s
 191.96.249.54:49217 169.125.128.112:80  ESTABLISHED 1s   901 B/s
 ...

¹ Setup: Ubuntu 14.04.5 LTS, Fail2Ban v0.8.11, tcptrack v1.4.2 ↑

About me

Hi, I'm Jonathan Experton.

I help companies start, plan, execute and deliver software development projects on time, on scope and on budget.

Montreal, Canada · GMT -4