Project41.View System Log Files

Project 41. View System Log Files

"How do I check whether Mac OS X is running smoothly behind the scenes and discover who's been hitting my Web site?"

This project considers log files, showing you where they are kept and how to use the tail command to view them in real time. It also discusses rotation, which stops log files from growing too big. Project 42 shows you how to view log files by using Apple's system log utility and how to control the system log daemon.

Know Your Log Files

Behind the scenes, your Mac runs many faceless applications. A faceless application runs in the background and has no direct user interface. Examples include many of the system components of OS X itself, the Apache Web server, and the firewall. Quite frequently, these applications have something to tell you, such as who's just hit your home page or that someone has tried to hack past the firewall. The applications have no way of presenting this information directly, and usually, the information is not of immediate concern anyway, so they write a report to a log file.

Take a look in directory /var/log , and you'll see several log files.

$

ls /var/log/*.log

/var/log/asl.log /var/log/lpr.log /var/log/system.log
...
/var/log/ipfw.log /var/log/secure.log

All log files are written to /var/log/ or a subdirectory within it. Here's a list of the most useful ones and what they report.

system.log. This is a general log file used by many system components. You'll see reports about network status, idiosyncratic behavior of applications, and indications of minor problems. This is a good place to look when you experience system hiccups.
secure.log. This log file tracks all attempts at authentication, both successful and unsuccessful .
ipfw.log. All attempts to access ports blocked by the firewall are logged here.
httpd/access_log. This log file records every page served by Apache.
httpd/error_log. This log file records Apache startup, shutdown, and error events.
mail.log. If you run the Postix mail server, it'll keep you informed from here.

Learn More

Project 21 shows you many ways to view text files, covering cat,less , and tail . Project 30 covers the nano text editor.

Let's take a look at Apache's log files, which are in the httpd subdirectory.

$

ls /var/log/httpd/*_log

/var/log/httpd/access_log /var/log/httpd/error_log

If you don't have these files, start Personal Web Sharing from System Preferences: Click Sharing, then click the Services tab, and check Personal Web Sharing. Launch Safari, and type the URL http://localhost/ . You should see a page saying "Seeing this instead of the website you expected?". We're not interested in the Web pagejust the log files Apache will have created.

View Log Files

Let's view some log files. They can get very big, so you may want to use an editor such as nano or browse them by using less . In this example, we'll use tail , which displays just the last ten lines of a file. Specify option -n followed by a number to display that number of lines instead of ten.

View the Apache log files by typing

$

tail /var/log/httpd/access_log

...
127.0.0.1 - - [07/Jul/2005:17:25:38 +0100] "GET / HTTP/...
127.0.0.1 - - [07/Jul/2005:17:25:39 +0100] "GET /apache...
127.0.0.1 - - [07/Jul/2005:17:25:40 +0100] "GET /favico...
$

tail /var/log/httpd/error_log

Processing config directory: /private/etc/httpd/users/*...
...
[Thu Jul 7 17:21:14 2005] [notice] Apache/1.3.33 (Dar...
[Thu Jul 7 17:21:14 2005] [notice] Accept mutex: floc...
[Thu Jul 7 17:25:40 2005] [error] [client 127.0.0.1] ...

You should see something like the lines above, including a record of your recent attempt to view the URL http://localhost/ . This project won't decipher specific log files; its purpose is to make you aware of them and give you some tips on viewing them.

Tip

Define a Bash function to tail a log file. Type

$

tlog () { tail ¬


-n${2:-15} ¬


/var/log/; }

Now you can simply type tlog system.log to view the last 15 lines of the system log or tlog system.log 30 to view the last 30 lines. Type tlog httpd/access_log to view the last 15 lines of Apache's access log .

The odd-looking variable expansion ${2:-15} says take the value of the second argument $2 if given; other-wise, use the value 15 .

Mac OS X also maintains a console log. It is written per user to /Library/Logs , not to the more usual /var/log directory. View your own by typing

$

tail /Library/Logs/Console/501/console.log

Replace 501 with your own UID (refer to Project 7 to learn about UIDs). To discover your own UID, type

$

id

Tip

Use grep ,, covered in Project 23, or sed , covered in Project 59, to search log files for specific information .

View Log Files in Real Time

Sometimes, we are interested in watching reports as they are written. Let's view a continually changing log file by using tail . Specify option -f , and tail won't exit after displaying the last few lines but will continue monitoring the log file, displaying each new line as it's appended.

$

tail -f -n3 /var/log/httpd/access_log

127.0.0.1 - - [07/Jul/2005:17:25:38 +0100] "GET / HTTP/...
127.0.0.1 - - [07/Jul/2005:17:25:39 +0100] "GET /apache...
127.0.0.1 - - [07/Jul/2005:17:25:40 +0100] "GET /favico...

If you still have the page http://localhost/ open , click the link "The Apache documentation has been included with this distribution," and watch the output from tail. You should see something like this.

127.0.0.1 - - [07/Jul/...] "GET /manual/ HTTP/1.1" 200 ...
127.0.0.1 - - [07/Jul/...] "GET /manual/images/apache_h...
127.0.0.1 - - [07/Jul/...] "GET /manual/images/pixel.gi...
127.0.0.1 - - [07/Jul/...] "GET /manual/images/index.gi...

Press Control-c to quit tail .

Log File Analyzers

Log files can be difficult to read and do not provide information in a statistical manner. They are, however, amenable to analysis by third-party applications. One such application is awstats , which generates Web, FTP, and mail statistics.

Find out more at http://awstats. sourceforge .net/ .

The Console Application

The Console application in Applications:Utilities:Console.app is the OS X-native equivalent of tail -f . Click Logs at the left end of the toolbar to reveal a list of log files.

Note

The system log file is rotated every day, and the last eight log files are archived.

Log File Rotation

Log files can get very big. To stop them from becoming excessively so, they are rotated and archived each week. The log file is zipped (compressed), and a new one is created. The zipped file is given the extension . 0.gz. A previously zipped file is kept and renamed . 1.gz. In fact, the last four log files are kept, as shown in the listing below. Each week, a new log file is created, and the old ones are shuffled down.

Let's confirm this by looking at the Apache access_log files for a server that's been running for more than a month.

$

cd /var/log/httpd/

$

ls access_log*

access_log      access_log.1.gz access_log.3.gz
access_log.0.gz access_log.2.gz access_log.4.gz

You may need to view a compressed log file. To avoid the uncompressviewdelete cycle, use gzcat or zless , both of which display compressed files directly. Functionally, gzcat is equivalent to cat , and zless is equivalent to less .

Let's view last week's Apache access_log file.

$

zless /var/log/httpd/access_log.0.gz

Press the spacebar to view the next page and q to quit zless .

Unfortunately, OS X doesn't include ztail , and pre-10.4 versions didn't include zless . You can simulate either of them by typing

$

gzcat /var/log/httpd/access_log.0.gz  tail -f

$

gzcat /var/log/httpd/access_log.0.gz  less

Learn More

Project 27 shows you how to compress and uncompress files.

Periodic Maintenance

Log files are rotated during periodic maintenance, which is run in the early hours of the morning and initiated by Apple's launchd (or by crontab in OS X versions before 10.4). If necessary, run the files manually, giving your administrator password when prompted. The following example will, among other things, rotate system.log , creating system.log.0.gz .

Tip

When the periodic maintenance scripts run, they add an audit trail to daily.out, weekly.out , and monthly.out , as appropriate. These files are in /var/log .