September 1, 2016
How to catch and block a brute force attack on Wordpress xmlrpc.php file
A few days ago I was investigating a database error on a Wordpress site that was just displaying Error establishing a database connection on every page.
After restarting the server, the problem was gone but came back quickly and showed a 500 error.
Find the problem
First thing I do on the server (Ubuntu 14.04) is a quick health check of resources with top:
$ top...PID USER VIRT RES SHR S %CPU %MEM TIME+ COMMAND885 nobody 303748 28380 20760 R 20.4 2.7 0:00.35 php-cgi894 nobody 302396 21136 14864 R 15.3 2.0 0:00.12 php-cgi865 nobody 307940 46408 34728 R 5.1 4.4 0:00.78 php-cgi869 nobody 307496 44924 33736 R 5.1 4.3 0:00.70 php-cgi870 nobody 306776 43232 32688 R 5.1 4.1 0:00.70 php-cgi874 nobody 305608 39896 30568 R 5.1 3.8 0:00.61 php-cgi889 nobody 301968 20508 14660 R 5.1 2.0 0:00.21 php-cgi890 nobody 303480 23932 16500 R 5.1 2.3 0:00.20 php-cgi897 nobody 300884 17500 12560 R 5.1 1.7 0:00.06 php-cgi...
Looks like PHP is eating all the server's resources which leaves other processes such as MySQL, Apache, or even the SSH server, starving.
Since PHP is only supposed to be called by the webserver, I want to check what's
going on ports 80 with tcptrack:
# install tcptrack first$ sudo apt-get update$ sudo apt-get install -y tcptrack# find network interface name$ ifconfig...venet0 Link encap:UNSPEC HWaddr ......# checking live traffic$ tcptrack -i venet0 port 80191.96.249.54:39074 167.114.88.210:80 ESTABLISHED 0s 902 B/s B/s191.96.249.54:50730 167.114.88.210:80 ESTABLISHED 1s 901 B/s B/s191.96.249.54:16331 167.114.88.210:80 ESTABLISHED 1s 900 B/s B/s
First column is the origin ip. It looks like somebody is hammering the server with web requests.
Next thing I want to check is webserver's log files to see what site and url is requested:
# what service is listening on port 80?$ sudo lsof -i :80 | grep LISTENapache2 11371 www-data 4u IPv6 3269827235 0t0 TCP *:http (LISTEN)# then checking apache logs$ sudo tail -100 /var/log/apache2/access.log191.96.249.54 - [26/Aug/2016:21:59:56 -0400] POST /xmlrpc.php...191.96.249.54 - [26/Aug/2016:21:59:57 -0400] POST /xmlrpc.php...191.96.249.54 - [26/Aug/2016:21:59:57 -0400] POST /xmlrpc.php...191.96.249.54 - [26/Aug/2016:21:59:57 -0400] POST /xmlrpc.php...191.96.249.54 - [26/Aug/2016:21:59:58 -0400] POST /xmlrpc.php...191.96.249.54 - [26/Aug/2016:21:59:58 -0400] POST /xmlrpc.php...191.96.249.54 - [26/Aug/2016:21:59:59 -0400] POST /xmlrpc.php...
After a quick search I understand it's a Wordpress brute force attack
on the xmlrpc.php file.
The automated attack is actually DoS-ing the server since it has only 1GB of RAM.
A targeted solution to the problem
First approach is to block the attacker's IP address with ufw:
# block ip$ sudo ufw deny from 191.96.249.54 to any# check if blocked$ tcptrack -i venet0 port 80191.96.249.54:39074 167.114.113.230:80 SYN_SENT 25s 0 B/s191.96.249.54:50730 167.114.113.230:80 SYN_SENT 26s 0 B/s
Now the server doesn't answer to the three-way handshake and connection will eventually timeout after 30 seconds.
A more future-proof solution
To prevent the attack from another IP address, one option is to block any request
to the xmlrpc.php file` from Apache's virtualhost config file:
$ sudo nano /etc/apache2/sites-available/000-default.conf<VirtualHost *:80>...<Files "xmlrpc.php">Require all denied</Files>...</VirtualHost>
The downside is that it doesn't prevent connections from hitting the webserver and still eat some resources if the 403 status code doesn't stop the attack.
Also, somebody might want to access the file for legitimate use...
A targeted, future-proof solution
To get the best of both solutions, I use Fail2ban
to watch log files and automatically block IP addresses based on custom rules.
I've posted another article on how to create custom Fail2ban rules.
