I have several scripts that scan various log files for signs of suspicious activity and block the offending IPs on my Web servers – pretty standard stuff. The trick, of course, is not to block yourself or someone you care about.

Here’s an example of what I am talking about:

The key here is the ${whitelist} containing IP addresses, I definitely don’t want to block no matter what. One way to generate such a whitelist is to scan the contents of /etc/hosts.allow and /etc/hosts. Because these two files can use either subnet or CIDR notation, we need to expand those into individual IPs. This should work OK if you don’t have any /8 or wider ranges.

I am extracting IPs and IP ranges from the two aforementioned config files in the example below. I am also using a netmast2cidr function to convert individual IPs and any ranges that use netmask notation to use CIDR notation. I then use nmap to expand the resulting ranges into a large list of individual IPs. And finally, I use the generated whitelist with my block command.

This works fine as long as you keep the whitelist relatively short. But what if you want to add all of the private networks (you know, ranges like 10.0.0.0/8) to your whitelist? All the private network ranges contain 34,668,544 IPs – this would make one hell of a pattern file for grep to parse.

The solution is to use grepcidr with a whitelist pattern file that uses CIDR notation. Here’s an example similar to the previous one, but with a couple of notable differences: I am adding the private networks, my external IP address (the wget bit below), I am no longer using nmap, and now I use grepcidr instead of grep.