10.8 Managing Log Files

There are several final suggestions we can make about log files. The first has to do with backups. We strongly recommend that you ensure that all of your log files are copied to your backup media on a regular basis, preferably daily. The timing of the backups should be such that any file that is periodically reset is copied to the backups before the reset is performed. This will ensure that you have a series of records over time to show system access and behavior.

Our second suggestion concerns how often to review the log files. We recommend that you do this at least daily. Keeping log records does you little service if you do not review them on a regular basis. Log files can reveal problems with your hardware, with your network configuration, and (of course) with your security. Consequently, you must review the logs regularly to note when a problem is actually present. If you delay for too long, the problem may become more severe; if there has been an intruder, he or she may have the time to edit the log files, change your security mechanisms, and do dirty deeds before you take notice.

Our third suggestion concerns how you process your log messages. Typically, log messages record nothing of particular interest. Thus, every time you review the logs (possibly daily, or several times a day, if you take our previous suggestion), you are faced with many lines of boring, familiar messages. The problem with this scenario is that you may become so accustomed to seeing this material that you get in the habit of making only a cursory scan of the messages to see if something is wrong, and this way you can easily miss an important message.

To address this problem, our advice is to filter the messages that you actually look at to reduce them to a more manageable collection. To do so requires some care, however. You do not want to write a filter that selects those important things you want to see and discards the rest. Such a system is likely to result in an important message being discarded without being read. Instead, you should filter out the boring messages, being as specific as you can with your pattern matching, and pass everything else to you to be read. Periodically, you should also study unfiltered log messages to be sure that you are not missing anything of importance.

Our last suggestion hints at our comments in Chapter 27, Who Do You Trust?, Who Do You Trust? Don't trust your logs completely! Logs can often be altered or deleted by an intruder who obtains superuser privileges. Local users with physical access or appropriate knowledge of the system may be able to falsify or circumvent logging mechanisms. And, of course, software errors and system errors may result in logs not being properly collected and saved. Thus, you need to develop redundant scanning and logging mechanisms: because something is not logged does not mean it didn't happen.

Of course, simply because something was logged doesn't mean it did happen, either - someone may cause entries to be written to logs to throw you off the case of a real problem or point a false accusation at someone else. These deceptions are easy to create with syslog if you haven't protected the network port from messages originating outside your site!