Finding the Locks Held by the Linux Kernel

We will now leave the theoretical discussion of the NLM and its interaction with the kernel lock arbitration methods and discuss system administration and maintenance activities as they relate to kernel and NLM lock arbitration.

To see what Posix fcntl and BSD flocks are currently granted by the kernel, examine the contents of the /proc/locks file. Although you can do this on any Linux system, this technique is perhaps most useful for troubleshooting locking problems on cluster nodes that have NFS-mounted data.

To view the current locks, first cat the /proc/locks kernel file:

 #cat /proc/locks 1  2      3         4     5     6  7  8       9 10  11       12       13       14       15 1: POSIX  ADVISORY  WRITE 14555 21:06:69890   0 EOF cd4f87f0 c02d9fc8 cd4f8570 00000000 cd4f87fc 2: POSIX  ADVISORY  WRITE 14005 21:06:69889   0 EOF cd4f87f0 cd4f873c cd4f8ca0 00000000 cd4f87fc 3: POSIX  ADVISORY  WRITE 8508  21:07:56232   0 EOF cd4f8f7c cd4f873c cd4f8908 00000000 cd4f8f88 4: FLOCK  ADVISORY  WRITE 12435 22:01:2387398 0 EOF cd4f8904 cd4f8f80 cd4f8ca0 00000000 cd4f8910 5: FLOCK  ADVISORY  WRITE 12362 22:01:1831448 0 EOF cd4f8c9c cd4f8908 cd4f83a4 00000000 cd4f8ca8 

Note 

I've added column numbers to this output to help make the following explanation easier to understand.

Column 1 displays the lock number and column 2 displays the locking mechanism (either flock or Posix fcntl). Column 3 contains the word advisory or mandatory; column 4 is read or write; and column 5 is the PID (process ID) of the process that owns the lock. Columns 6 and 7 are the device major and minor number of the device that holds the lock,^[12] while column 8 is the inode number of the file that is locked. Columns 9 and 10 are the starting and ending byte range of the lock, and the remaining columns are internal block addresses used by the kernel to identify the lock.

To learn more about a particular lock, use lsof to examine the process that holds it. For example, consider this entry from above:

 1: POSIX ADVISORY WRITE 14555 21:06:69890 0 EOF cd4f87f0 c02d9fc8 cd4f8570 00000000 cd4f87fc 

The PID is 14555, and the inode number is 69890. (See boldface text.) Using lsof, we can examine this PID further to see which process and file it represents:

 #lsof -p 14555 

The output of this command looks like this:

 COMMAND   PID    USER   FD   TYPE   DEVICE    SIZE    NODE    NAME lockdemo  14555  root   cwd  DIR    33,6      2048    32799   /root lockdemo  14555  root   rtd  DIR    33,6      2048    2       / lockdemo  14555  root   txt  REG    33,6      14644   69909   /tmp/lockdemo lockdemo  14555  root   mem  REG    33,6      89547   22808   /lib/ld-2.2.5.so lockdemo  14555  root   mem  REG    33,6      1401027 26741   /lib/i686/libc-2.2.5.so lockdemo  14555  root   0u   CHR    136,3             5       /dev/pts/3 lockdemo  14555  root   1u   CHR    136,3             5       /dev/pts/3 lockdemo  14555  root   2u   CHR    136,3             5       /dev/pts/3 lockdemo  14555  root   3u   REG    33,6      1024    69890   /tmp/mydata 

The last line in this lsof report tells us that the inode 69890 is called /tmp/ mydata and that the name of the command or program that is locking this file is called lockdemo.^[13]

To confirm that this process does, in fact, have the file open, enter:

 #fuser /tmp/mydata 

The output gives us the file name followed by any PIDs that have the file open:

 /tmp/mydata: 14555 

To learn more about the locking methods used in NFSv3, see RFC 1813, and for more information about the locking methods used in NFSv4, see RFC 3530. (Also see the Connectathon lock test program on the "Connectathon Test Suites" link at http://www.connectathon.org.)

^[12]You can see the device major and minor number of a filesystem by examining the device file associated with the device (ls -l /dev/hda2, for example).

^[13]The source code for this program is currently available online at http://www.ecst.csuchico.edu/~beej/guide/ipc/flock.html.