Linux SysV IPC Resources and the ipcs Command

Linux supports the three types of SysV IPC resources. The Linux kernel default configuration normally enables SysV IPC support. If the SysV IPC is not supported in your Linux kernel, you need to configure your kernel source with the CONFIG_SYSVIPC set to enable SysV IPC facilities. You need to rebuild your kernel after changing the configuration.

The SysV IPC facilities are widely used throughout various Linux applications. For example, some database applications create shared memory segments for data sharing. The Linux kernel dynamically allocates memory for IPC objects whenever a new IPC resource is requested. However, due to the amount of memory consumed by the kernel for these IPC resources, the kernel enforces limits on the usage of IPC resources. Some of the limits can be tuned dynamically. When tuning parameters to meet your IPC resource needs, it is important that you consider memory usage consumed by the resources.

Pipes, on the other hand, are so fundamental to UNIX system operation that they are always available in the kernelthey are not configurable into or out of the kernel. Typically, pipes are used in shell commands to direct output from one process into another process. For example:

 bash> ls lsR | grep linux 

Pipes differ from the SysV IPC in that they require processes to be related. Pipes are created in pairsthe first element for reading and the second element for writing. During a fork, the pipes can then be handed off between the parent and the various child processes, and henceforth be used to communicate between these related processes.

FIFOs (named pipes) are created as special files in the file system using the mkfifo filename command.

The ipcs Command

Almost all Linux distributions include the ipcs command, which provides information about the IPC resources that are currently loaded on the system. ipcs lets you determine the current IPC limits that the system allows and also lets you check the status of the three IPC resources that are currently in use on the system. For example, if your application fails to start, you can check the IPC usage on your system to determine if an IPC limit has been exceeded. To determine the status of the system's IPC resources, as root, issue the ipcs command with the u option:

 # ipcs -u ------ Shared Memory Status -------- segments allocated 32 pages allocated 2361 pages resident  253 pages swapped   982 Swap performance: 0 attempts     0 successes ------ Semaphore Status -------- used arrays = 128 allocated semaphores = 256 ------ Messages: Status -------- allocated queues = 0 used headers = 0 used space = 0 bytes 

To determine the limits on the IPC resources reported by the ipcs u command, use the ipcs l command:

 # ipcs -l ------ Shared Memory Limits -------- max number of segments = 4096 max seg size (kbytes) = 32768 max total shared memory (kbytes) = 8388608 min seg size (bytes) = 1 ------ Semaphore Limits -------- max number of arrays = 128 max semaphores per array = 250 max semaphores system wide = 32000 max ops per semop call = 32 semaphore max value = 32767 ------ Messages: Limits -------- max queues system wide = 16 max size of message (bytes) = 8192 default max size of queue (bytes) = 16384 

This output shows that the system has hit the limit of the maximum number of semaphore arrays, or semaphore sets. This limit could be resolved by increasing the value of kernel parameter semmni, which defines the total number of semaphore sets a system can have. Linux supports dynamically resizing most kernel IPC parameters, as well as changing them in a static way.

IPC Identifiers and Their Limits

In the Linux kernel, each IPC resource is described as a data structure in the form of an IPC identifier or an IPC object. The IPC mechanism is implemented based on this data structure and the operations associated with it. The Linux kernel includes a set of system calls to create and manipulate the IPC identifiers and objects.

An IPC identifier is identified by a unique key for its type (either semaphores, message queues, or shared memory). Each type of the IPC resource is created by its corresponding "create" system call: semget() for semaphores, msgget() for message queues, or shmget() for shared memory. These *get() system calls return a unique IPC ID for the caller. After the application gets the ID, it can reference the IPC object via the ID and control the IPC identifier by the corresponding "control" system calls: semctl() for semaphores, msgctl() for message queues, and shmctl() for shared memory.

Additionally, the IPC mechanism in the Linux kernel provides system calls that allow an application to manipulate IPC resources: semop() increases or decreases semaphore values, msgsnd() and msgrcv() send and receive messages between processes via message queues, and shmat() and shmdt()are for shared memory operations and allow a process to attach or detach itself to or from a shared memory segment.

Each IPC identifier is of type struct ipc_ids, which is of size 44 bytes. Each IPC resource type contains an array to store pointers to the identifiers. The total number of entries in the global array is determined by the maximum number of identifiers of the corresponding IPC resource type (semmni, msgmni, or shmmni). You can resize the array by tuning the related kernel parameters. However, there is an upper limit for these parameters: IPCMNI. IPCMNI defines the maximum number of IPC identifiers system-wide for each IPC resource type. When the Linux kernel tries to resize the semaphore identifier array according to the new value of semmni (the maximum number of semaphore identifiers), it also checks whether the new size is over the IPCMNI limit. The default value of IPCMNI is 32767, and this parameter cannot be tuned at runtime. The IPCMNI parameter applies to all three IPC resources.