Hack 20 Deal with Disk Hogs


figs/expert.gif figs/hack20.gif

Fortunately, you no longer have to be a script guru or a find wizard just to keep up with what is happening on your disks.

Think for a moment. What types of files are you always chasing after so they don't waste resources? Your list probably includes temp files, core files, and old logs that have already been archived. Did you know that your system already contains scripts capable of cleaning out those files? Yes, I'm talking about your periodic scripts.

2.9.1 Periodic Scripts

You'll find these scripts in the following directory on a FreeBSD system:

% ls /etc/periodic/daily | grep clean 100.clean-disks 110.clean-tmps 120.clean-preserve 130.clean-msgs 140.clean-rwho 150.clean-hoststat

Are you using these scripts? To find out, look at your /etc/periodic.conf file. What, you don't have one? That means you've never tweaked your default configurations. If that's the case, copy over the sample file and take a look at what's available:

# cp /etc/defaults/periodic.conf /etc/periodic.conf # more /etc/periodic.conf

2.9.1.1 daily_clean_disks

Let's start with daily_clean_disks. This script is ideal for finding and deleting files with certain file extensions. You'll find it about two pages into periodic.conf, in the Daily options section, where you may note that it's not enabled by default. Fortunately, configuring it is a heck of a lot easier than using cron to schedule a complex find statement.

Before you enable any script, test it first, especially if it'll delete files based on pattern-matching rules. Back up your system first!

For example, suppose you want to delete old logs with the .bz2 extension. If you're not careful when you craft your daily_clean_disks_files line, you may end up inadvertently deleting all files with that extension. Any user who has just compressed some important data will be very miffed when she finds that her data has mysteriously disappeared.


Let's test this scenario. I'd like to prune all .core files and any logs older than .0.bz2. I'll edit that section of /etc/periodic.conf like so:

# 100.clean-disks daily_clean_disks_enable="YES"                     # Delete files daily daily_clean_disks_files="*.[1-9].bz2 *.core"       # delete old logs, cores daily_clean_disks_days=1                           # on a daily basis daily_clean_disks_verbose="YES"                    # Mention files deleted

Notice my pattern-matching expression for the .bz2 files. My expression matches any filename (*) followed by a dot and a number from one to nine (.[1-9]), followed by another dot and the .bz2 extension.

Now I'll verify that my system has been backed up, and then manually run that script. As this script is fairly resource-intensive, I'll do this test when the system is under a light load:

# /etc/periodic/daily/100.clean-disks Cleaning disks: /usr/ports/distfiles/MPlayer-0.92.tar.bz2 /usr/ports/distfiles/gnome2/libxml2-2.6.2.tar.bz2 /usr/ports/distfiles/gnome2/libxslt-1.1.0.tar.bz2

Darn. Looks like I inadvertently nuked some of my distfiles. I'd better be a bit more explicit in my matching pattern. I'll try this instead:

# delete old logs, cores daily_clean_disks_files="messages.[1-9].bz2 *.core"        # /etc/periodic/daily/100.clean-disks Cleaning disks: /var/log/messages.1.bz2 /var/log/messages.2.bz2 /var/log/messages.3.bz2 /var/log/messages.4.bz2

That's a bit better. It didn't delete /var/log/messages or /var/log/messages.1.bz2, which I like to keep on disk. Remember, always test your pattern matching before scheduling a deletion script. If you keep the verbose line at YES, the script will report the names of files it deletes.

2.9.1.2 daily_clean_tmps

The other cleaning scripts are quite straightforward to configure. Take daily_clean_tmps, for example:

# 110.clean-tmps daily_clean_tmps_enable="NO"                   # Delete stuff daily daily_clean_tmps_dirs="/tmp"                   # Delete under here daily_clean_tmps_days="3"                      # If not accessed for daily_clean_tmps_ignore=".X*-lock quota.user quota.group" # Don't delete                                                           # these daily_clean_tmps_verbose="YES"                 # Mention files deleted

This is a quick way to clean out any temporary directories. Again, you get to choose the locations of those directories. Here is a quick way to find out which directories named tmp are on your system:

# find / -type d -name tmp /tmp /usr/tmp /var/spool/cups/tmp /var/tmp

That command asks find to start at root (/) and look for any directories (-type d) named tmp (-name tmp). If I wanted to clean those daily, I'd configure that section like so:

# 110.clean-tmps # Delete stuff daily daily_clean_tmps_enable="YES"                         daily_clean_tmps_dirs="/tmp /usr/tmp /var/spool/cups/tmp /var/tmp"         # If not accessed for daily_clean_tmps_days="1"                             # Don't delete these daily_clean_tmps_ignore=".X*-lock quota.user quota.group"  # Mention files deleted daily_clean_tmps_verbose="YES"

Again, I immediately test that script after saving my changes:

# /etc/periodic/daily/110.clean-tmps Removing old temporary files:   /var/tmp/gconfd-root

This script will not delete any locked files or temporary files currently in use. This is an excellent feature and yet another reason to run this script on a daily basis, preferably at a time when few users are on the system.

2.9.1.3 daily_clean_preserve

Moving on, the next script is daily_clean_preserve:

# 120.clean-preserve daily_clean_preserve_enable="YES"              # Delete files daily daily_clean_preserve_days=7                    # If not modified for daily_clean_preserve_verbose="YES"             # Mention files deleted

What exactly is preserve? The answer is in man hier. Use the manpage search function (the / key) to search for the word preserve:

# man hier /preserve        preserve/ temporary home of files preserved after an accidental                   death of an editor; see (ex)1

Now that you know what the script does, see if the default settings are suited for your environment. This script is run daily, but keeps preserved files until they are seven days old.

The last three clean scripts deal with cleaning out old files from msgs, rwho and sendmail's hoststat cache. See man periodic.conf for more details.

Incidentally, you don't have to wait until it is time for periodic to do its thing; you can manually run any periodic script at any time. You'll find them all in subdirectories of /etc/periodic/.

2.9.2 Limiting Files

Instead of waiting for a daily process to clean up any spills, you can tweak several knobs to prevent these files from being created in the first place. For example, the C shell itself provides limits, any of which are excellent candidates for a customized dot.cshrc file [Hack #9].

To see the possible limits and their current values:

% limit cputime         unlimited filesize        unlimited datasize        524288 kbytes stacksize       65536 kbytes coredumpsize    unlimited memoryuse       unlimited vmemoryuse      unlimited descriptors     4557  memorylocked    unlimited maxproc         2278  sbsize          unlimited

You can test a limit by typing it at the command line; it will remain for the duration of your current shell. If you like the limit, make it permanent by adding it to .cshrc. For example:

% limit filesize 2k % limit | grep filesize filesize     2 kbytes

will set the maximum file size that can be created to 2 KB. The limit command supports both k for kilobytes and m for megabytes. Do note that this limit does not affect the total size of the area available to store files, just the size of a newly created file. See the Quotas section of the FreeBSD Handbook if you intend to limit disk space usage.

Having created a file limit, you'll occasionally want to exceed it. For example, consider decompressing a file:

% uncompress largefile.Z Filesize limit exceeded % unlimit filesize % uncompress largefile.Z %

The unlimit command will allow me to override the file-size limit temporarily (for the duration of this shell). If you really do want to force your users to stick to limits, read man limits.

Now back to shell limits. If you don't know what a core file is, you probably don't need to collect them. Sure, periodic can clean those files out for you, but why make them in the first place? Core files are large. You can limit their size with:

limit coredumpsize 1m

That command will limit a core file to 1 MB, or 1024 KB. To prevent core files completely, set the size to 0:

limit coredumpsize 0

If you're interested in the rest of the built-in limits, you'll find them in man tcsh . Searching for coredumpsize will take you to the right spot.

2.9.3 The Other BSDs

The preceding discussion is based on FreeBSD. Other BSD systems ship with similar scripts that do identical tasks, but they are kept in a single file instead of in a separate directory.

2.9.3.1 NetBSD

For daily, weekly, and monthly tasks, NetBSD uses the /etc/daily, /etc/weekly, and /etc/monthly scripts, whose behavior is controlled with the /etc/daily.conf, /etc/weekly.conf, and /etc/monthly.conf configuration files. For more information about them, read man daily.conf, man weekly.conf, and man monthly.conf.

2.9.3.2 OpenBSD

OpenBSD uses three scripts, /etc/daily, /etc/weekly, and /etc/monthly. You can learn more about them by reading man daily.

2.9.4 See Also

  • man periodic.conf

  • man limits

  • man tcsh

  • The Quotas section of the FreeBSD Handbook (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/quotas.html)



BSD Hacks
BSD Hacks
ISBN: 0596006799
EAN: 2147483647
Year: 2006
Pages: 160
Authors: Lavigne

Similar book on Amazon

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net