Low- to No-Cost Methods to Review Webserver Logs for Potential Security Issues

This is a description of the inexpensive methods I devised to extract and tally records of interest in order to analyze webserver logfiles for potential security problems, compromise attempts, while also obtaining IP address statistics. The tools used are the Unix fgrep utility and Analog, the freely distributed logfile analysis tool for multiple platforms. These techniques can easily be adapted to the logfiles produced by almost any platform and use other text extraction utilities than fgrep. If implemented on another site, the adaptation process will require some analysis of the logfiles of a given site as well as additional customization to eliminate 'false positives' without the introduction of 'false negatives.' The purpose of this monitoring on my site was to observe the compromise attempts used on the site and replicate them on the internal development system which mirrors the internet-facing production site. This replication should verify whether the attempt had any harmful effects on the production platform. This development platform runs on a server whose hardware and software are the same as the production webserver with no connection to the internet. It is used as a 'quality assurance' server in the final stage of pre-production testing.