I Want to Know What s Happening

I Want to Know What's Happening

The fundamental motivation for logging data is that someone wants to know something about the system. Table 14-1 captures what people want to know according to category, purpose, and audience.

Developers often misuse log files by storing one category of data in a log designed for anotherdebugging information in the operational status log file, say, or behavioral data in the system configuration log file. These mistakes, whether intentional or not, dilute the value of the data, invalidate or seriously complicate postprocessing analysis, and contribute to overall system complexity.

Table 14-1. Log File Motivations and Audiences

Category	Purpose	Audience
Debugging/Error Logs	Provide information on system operation as an aide in debugging.	During construction, developers are the primary audience. After release, technical support and professional services personnel can use this information to resolve problems in the field.
Error Recovery	Capture information that can be used to restore the working state of a system in case something crashes.	Almost exclusively the domain of the application. An example is a database management system transaction log.
Performance Tuning	Provide information on one or more aspects of performance, usually with the goal of improving it.	Professional services organizations, customer IT organization sites, and end users.
Capacity Planning	Provide information on actual resources consumed during system operation.	Customer IT organizations and end users.
Behavior Tracking and Auditing	Provide information on how various actors (end users, system components , other systems) interact with the component in question.	Marketing organizations (for user /feature profiling) and security organizations (for audit trails). In these cases the content of the log may be similar but how the log is used is different.
System Configuration Management	Provide information on the system context. These logs are often symmetric with configuration files.	As a subset of debugging information, it has the same audience.
Operational Status	Provide information on the current operational status of the system (e.g., how many transaction requests are in the queue, how many users are connected to it).	IT and professional services organizations.

No Categories Are Better than Two

Log data categories are not hard boundaries. Sometimes you're just not certain where the data should be stored because it could be used for more than one purpose or you're not certain who will be using the data or how. One option for handling this situation is to store log data in a single log file that has appropriate identifiers for each type of log event. Postprocessing tools can then read only one file to obtain the data they need.

In one application I worked on the system administrator specified in a configuration file the locations of various files required at initialization and used during operation. As the system processed these files, it logged the results. These log data were used in several different ways.

• Developers used them while creating and debugging the system.

• Technical support used them to help customers recover from a system crash.

• Professional services used them for performance tuning and capacity planning.

We didn't spend a lot of time debating where to store these data. Instead, we just put them in a single file.

A more serious misuse of log files is when a developer uses them for storing nonlogging data. The most flagrant abuse I ever experienced was when one developer used a log file to store persistent data! A good rule of thumb is that log files be read only, in that one system generates log data that other systems consume . There are exceptions to this rule, such as when a system can process its own log data (I've worked on systems that can self-tune key performance parameters based on log file analysis), but they are fairly rare. Application processing of log files should be completely optional: If someone deletes a log file and your application crashes, you've got a design problem that needs to be fixed.