Logtop is an awesome, albeit a little quirky, real-time log analysis tool developed by Julien Palard. You should use logtop when time is of the essence. When you cannot wait for your cron job to run to analyze log files from last night. When you need to know if you’re being hacked now – not yesterday.

Here’s how I installed it on CentOS 6:

And here’s a very basic example of how to use it. In this case I am counting the number of hits by individual IPs against by httpd server. This can be useful if you need to see who needs to be firewalled right now.

Note the fflush syntax allowing awk to flush its buffers in real time and thus be usable by toplog. The end result is simple: toplog counts frequency and number of occurrences of whatever you’re piping into it.

Here’s a slightly more advanced variation of the above example. It will find and watch multiple access_log files and show the hit frequency for each IP that is not on your private network. If you think someone out there is killing your server with requests, this is how you find them.

Seeing the IPs is nice, but it would be useful to see a bit more information about each IP. If you install geoiplookup, you can modify the command above to make use of it:

The output might look something like this:

Another example shows how random /dev/random is:

Here’s an example showing DHCP server activity by call type. Note the --line-buffered option for the grep command.

The thing to understand about logtop is that the tool is designed for real-time analysis. You can’t use it to produce a frequency report of historical log data, even if this data contains timestamps. So, real-time analysis – logtop. Historical data analysis – something else (and there’re plenty of choices here).

Things get a little tricky if you need to run logtop against real-time data for a defined period of time. Here’s an example of running the command for one minute:

Here’s a slightly more advanced example that will use iptables to block IPs that accessed your httpd at a rate of greater than 0.9 times per second over the past minute, Something like this can be useful to fend off a DOS attack or an overly aggressive Web crawler.

 

Leave A Reply

Please enter your comment!
Please enter your name here