Hack92.Find Out Why You Can t Unmount a Partition


Hack 92. Find Out Why You Can't Unmount a Partition

If you can't unmount a disk because it's busy, you can use the lsof and fuser commands to find open files or pesky attached processes.

The popularity of removable drives and their usability for things such as backups [Hack #50] makes mounting and unmounting partitions a fairly common activity while a system is running. Another not-so-common but more critical sysadmin activity is the need to unmount a drive in an emergency, such as when one of your users has accidentally deleted his thesis or the source code for your next-generation product, or the disk begins getting write errors and you need to initiate recovery ASAP. In either case, it's truly irritating when you can't unmount a partition because some unknown process is using it in one way or another. Shutting down a system just to unmount a disk so that you can remove or repair it is clearly overkill. Isn't there a better way? Of course there isread on.

10.5.1. Background

One of the most basic rules of Linux/Unix is that you can't unmount a partition while a process is writing to or running from it. Trying to do so returns an informative but fairly useless message like the following:

 $ sudo umount /mnt/music umount: /mnt/music: device is busy umount: /mnt/music: device is busy 

In some cases, terminating the processes associated with a partition is as easy as looking through all your windows for suspended or background processes that are writing to the partition in question or using it as their current working directory and terminating them. However, on multi-user, graphical systems with many local and remote users, this isn't always as straightforward as you'd like.

As progress toward an ultimate solution to this frustration, special-purpose Linux specifications such as Carrier-Grade Linux (CGL) require some "forced unmount" functionality in the kernel (http://developer.osdl.org/dev/fumount/), and the umount command includes a force (-F) option for NFS filesystems. That's all well and good, but those of us who are using vanilla Linux distributions on local disks still need a practical solution that doesn't require patching each kernel or killing a fly with a hammer through an immediate shutdown.

Recent versions of the umount command provide a l option to "lazily" unmount a filesystem immediately, and then try to clean up references to the filesystem as the processes associated with them terminate. This is certainly interesting and can be useful, but I generally prefer to know what's going on if I can't unmount a filesystem that I think I should be able to unmount. Your mileage may vary.


Linux provides two commands that you can use to identify processes running on a filesystem so that you can (hopefully) terminate them in one way or another: fuser (find user process) and lsof (list open files). The key difference between the two is that the fuser command simply returns the process IDs (PIDs) of any processes associated with the file or directory specified as an argument, while the lsof command returns a full process listing that provides a variety of information about the processes associated with its argument(s). Both are quite useful, and which you use is up to you. The next two sections show how to use each of these commands to help find the pesky process(es) that are keeping you from unmounting a partition.

The Open Source Development Lab's forced unmount page, referenced at the end of this hack, provides a cool but crude script called funmount that tries to automatically combine a number of passes of fuser with the appropriate unmount commands to "do the right thing" for you when you need to forcibly unmount a specified partition. It's worth a look.


10.5.2. Finding Processes That Are Using a Filesystem

The fuser command returns the PIDs of all the processes associated with the device or mounted filesystem that is specified as an argument, along with terse information that summarizes the way in which each process is using the filesystem. To search for all processes associated with a mounted filesystem or device, you need to specify the m option, followed by the name of the filesystem or its mount point. For example, the following fuser command looks for processes associated with the filesystem mounted at /mnt/music on my system:

 $ fuser -m /mnt/music /mnt/music: 29846c 31763c 

Each process ID returned by the fuser command is followed by a single letter that indicates how the specified process is using the filesystem. The most common of these is the letter c, which indicates that the process is using a directory on that filesystem as its current working directory. In the previous example, you can see that both of the processes listed are using the filesystem as their current working directory.

Once you have this sort of output, you can use the grep command to search for each of the specified process IDs and see what they're actually doing, as in the following example:

 $ ps alxww | grep 29846 0 1000 29846 7797 16  0 9992 2284 wait Ss    pts/13 0:00 /bin/bash 4    0 29912 29846 16 0 24608 1364 finish T  pts/13 0:00 su 0 1000 31763 29846 16 0 10292 2480 -   S+  pts/13 0:00 vi playlist.m3u 0 1000 31789 30009 17 0 3788 764 -     R+ pts/14 0:00 grep -i 29846 

By default, the fuser command returns all active processes. However, as we can (accidentally) see in the above process listing, there is also a terminated su process that is a child of the process that fuser identified, which could prevent us from unmounting the filesystem in question. To provide a more complete fuser output listing, you should generally run the fuser command as root (or via sudo), and also specify the a option to ensure that all processes are listed, regardless of their states, as in the following example:

 $ sudo fuser -am /dev/mapper/data-music /dev/mapper/data-music: 29846c 29912c 29916c 31763c 32088 

As you can see, fuser now picks up the process ID of the su process.

If you're really in a hurry, you can also specify the fuser command's k option, which kills any processes it finds. It's generally a good idea to try to find the processes in question and terminate them cleanly, but in some cases you may just want to kill the processes as quickly as possible (for example, when you're hoping to subsequently recover deleted files and want to prevent filesystem updates).


10.5.3. Listing Open Files

The fuser command returns PIDs that require subsequent interpretation to figure out which files they're actually using on the specified filesystem (though the status indicator appended to each PID gives you a quick idea of how each process is using the filesystem). In contrast, the lsof command returns more detailed information about processes that have open files on a specified filesystem, and may tell you everything that you need to know in one swell foop. For example, the following is the output of the lsof command on the same filesystem used in the previous examples:

 $ lsof /mnt/music COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME bash 29846 wvh cwd DIR 253,0    64 131 /mnt/music/test vi   31763 wvh cwd DIR 253,0    64 131 /mnt/music/test vi   31763 wvh 4u  REG 253,0 12288 133 /mnt/music/test/.playlist. m3u.swp 

The first column (COMMAND) shows each command that the system is running that is associated with the file, directory, or mount point that you specified as an argument. The last column (NAME) identifies the file or directory that each command is actually associated with. The FD column shows the active file descriptors associated with the process or, in the case of a shell or command, the fact that the shell or command is using the specified directory as its current working directory (cwd).

As with fuser, when run by a standard user the output of lsof shows only active processes. To get more complete output, you should generally run the lsof command as root (or via sudo), as in the following example:

 $ sudo lsof /mnt/music COMMAND PID   USER FD  TYPE DEVICE SIZE  NODE NAME bash    29846  wvh cwd DIR   253,0    64 131  /mnt/music/test su 29912 root cwd DIR   253,0    17 128  /mnt/music bash 29916 root cwd DIR  253,0    17 128  /mnt/music vi 31763  wvh cwd DIR   253,0    64 131  /mnt/music/test vi 31763  wvh  4u REG  253,0 12288 133  /mnt/music/test/.playlist. m3u.swp 

You can see that the output of this instance of the lsof command picked up the suspended su process, and also identifies the bash shell associated with this process.

Unlike the fuser command, the lsof command doesn't provide an option to automatically terminate the processes it has located, but it provides a good deal more information to begin with. Once you know exactly what they're doing and are sure that it's safe to kill them, you can always quickly terminate each process manually from the command line in order to unmount the filesystem.

10.5.4. Summary

The fuser and lsof commands are useful additions to your Linux sysadmin toolset. fuser quickly delivers information about active processes and provides an option to automatically and instantly terminate processes associated with the filesystems or files that you specify as arguments, but its output requires subsequent interpretation (if you have time to play detective). The lsof command returns more detailed information about the associated processes (although additional interpretation may still be required), and can also display information about network-related files and sockets that may be open (see its manpage or FAQ for more details). However, it doesn't include an option to quickly terminate all of the processes in one go. In my experience, fuser is faster, but lsof provides a much richer spectrum of information. Each is useful at different times, depending on what you're looking for and how quickly you need to find (and perhaps kill) it.

10.5.5. See Also

  • http://www.osdl.org/lab_activities/carrier_grade_linux

  • http://developer.osdl.org/dev/fumount/

  • ftp://lsof.itap.purdue.edu/pub/tools/unix/lsof/FAQ



Linux Server Hacks (Vol. 2)
BSD Sockets Programming from a Multi-Language Perspective (Programming Series)
ISBN: N/A
EAN: 2147483647
Year: 2003
Pages: 162
Authors: M. Tim Jones

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net