Section 5.15. Interprocess Communications

5.15. Interprocess Communications

Semaphores, message queues, and shared memory make up the Interprocess Communications (IPC) facility available in AIX. A semaphore is synchronization primitive, a message queue is a kernel-resident linked list of messages, and shared memory is memory that is shared between processes mapped into the address space of the process sharing the memory.

5.15.1. Semaphores

There are two forms of synchronization mechanisms within the UNIX world. One is the System V IPC version, and the other is POSIX 1003.1b. System V semaphores allows for semaphore sets, meaning that a unique semaphore identifier may contain multiple semaphores. Whether a semaphore identifier contains one semaphore or a set of semaphores, each is determined when the semaphore is created with the semget system call. The second argument to semget determines the number of semaphores that will be associated with the semaphore identifier returned by semget. The semctl() or semop() system allows the programmer to control and operate on the semaphore set as needed.

The POSIX 1003.1b semaphore mechanism allows programmers to create individual named and unnamed semaphores. The basic operation goes like this. The semaphore is created with the sem_init(). Once created, the semaphore can be used to synchronize access to shared resources by forcing the calling thread or process to wait on the semaphore before entering the critical section by calling sem_wait(). After the calling process is finished with the shared resource, it signals to the other processes or threads waiting that it is finished by posting the semaphore, sem_post(). These semaphores are ideal when resources must be shared between either two or more distinct processes (named semaphores) or individual threads within the same process (unnamed semaphore).

AIX and Linux support both mechanisms. Table 5-11 gives a side-by-side comparison between AIX and Linux.

Table 5-11. AIX and Linux IPC API Comparison: POSIX Semaphores

AIX	Linux	AIX ERRNO(s)	Linux ERRNO(s)	Description
sem_init	sem_init	EFAULT EINVAL ENFILE ENOMEM ENOSPC ENOTSUP	EINVAL ENOSYS	Initializes the semaphore object.
sem_wait	sem_wait	EACCES EAGAIN (semwait only) EFAULT EIDRM	Does not set ERRNO. Always returns 0.	Suspends the calling thread until the semaphore pointed to by sem has a nonzero count.
sem_trywait	sem_trywait	EINTR EINVAL ENOMEM ENOTSUP	EAGAIN
sem_post	sem_post	EACCES EFAULT EIDRM EINVAL ENOMEM ENOTSUP	ERANGE	Unlocks a semaphore.
sem_getvalue	sem_getvalue	EACCES EFAULT EINVAL ENOMEM ENOTSUP	Does not set ERRNO Always returns 0.	Gets the value of a semaphore.
sem_destroy	sem_destroy	EACCES EFAULT EINVAL ENOTSUP	EBUSY	Destroys a semaphore object, freeing the resources it might hold.
sem_timedwait	sem_timedwait	EFAULT EINVAL ETIMEDWAIT EDEADLK ENTR	EENVAL ETIMEDWAIT EDEADLK ENTR	Advanced real-time wait.
sem_unlink	sem_unlink	EACCES EFAULT ENAMETOOLONG ENOENT ENOTSUP	EACCES ENAMETOLONG ENOENT	Removes a named semaphore.
sem_close	sem_close	EFAULT EINVAL ENOMEM ENOTSUP	EINVAL	Closes a named semaphore.
sem_open	sem_open	EACCES EEXIST EFAULT EINVAL EMFILE ENAMETOOLONG ENOMEM ENOTSUP ENOSPC	EACCES EEXIST EINTR EENVAL EMFILE ENAMETOOLONG ENFILE ENOENT ENOSPC	Opens a named semaphore.

Semaphores are the primitive synchronization mechanism in many operating systems. The APIs mentioned are standard within the UNIX community. Semaphore sets are also a standard within UNIX, as discussed in detail in the next section.

5.15.2. System V Semaphore sets

The System V version of semaphore functions are semget(), semctl(), and semop(). They perform operations on sets of semaphores associated with a predefined identifier. In Linux, the semget(), semctl(), and semop() functions are completely compatible with AIX versions. The only noticeable difference is in semop(). In AIX, the semop() sets the ENOSPC ERRNO and in Linux it does not. However, this ERRNO is defined in Linux, which means no coding changes are necessary when migrating to Linux because any code that handles this error in AIX will be ignored in Linux.

5.15.3. Message Queues (mqueue.h)

Both AIX and Linux provide fully POSIX.1b-compliant message queues. On Linux prior to version 2.6.6.rtl, message queues were implemented in a separate required linkable library (libmqueue) that was shipped for certain distributions of Linux. In Linux kernel version 2.6.6.rtl, the Linux team decided to include the libmqueue library in the glibc 2.3.4 (04/08/2004 in changelog). Although both AIX and Linux provide support for message queues, the ERRNOs they return for certain errors sometimes differ. Minor changes may be necessary in your code if your application takes certain actions based on these ERRNOs. Table 5-12 compares the ERRNO differences between AIX and Linux.

Table 5-12. AIX and Linux IPC API Comparison: Message Queues

AIX->Linux	AIX ERRNO(s)	Linux ERRNO(s)	Description
mq_close	EBADF ENOMEM ENOTSUP	EBADF EBUSY	Closes a message queue
mq_open	EACCES EEXIST EFAULT EINVAL ENOMEM ENOSPC ENOTSUP ENOENT ENFILE ENAMETOOLONG	EACCES EEXIST EINTR EINVAL EMFILE ENOSPC ENOENT ENFILE ENAMETOOLONG	Opens a message queue
mq_notify	EBADF EBUSY EFAULT ENOMEM ENOTSUP EINVAL	EBADF EBUSY	Notifies a process that a message is available
mq_getattr	EBADF EFAULT ENOMEM EINVAL ENOTSUP	EBADF	Gets message queue attributes
mq_unlink	EACCES EFAULT EINVAL ENAMETOOLONG ENOTSUP	EACCES ENOENT ENAMETOOLONG	Removes a message queue
mq_send	EAGAIN EFAULT EBADF EIDRM EINTR EINVAL EMSGSIZE ENOMEM ENOTSUP	EAGAIN EBADF EINTR EINVAL EMSGSIZE	Sends a message to the message queue
mq_setattr	EBADF EFAULT EINVAL ENOMEM ENOTSUP	EBADF	Sets message queue attributes
mq_send, mq_timedsend	EAGAIN EBADF EFAULT EIDRM EINTR EINVAL EMSGSIZE ENOTSUP ETIMEDOUT	EAGAIN EBADF EINTR EINVAL EMSGSIZE ETIMEDOUT	Sends a message to a message queue (real time)
mq_receive	EAGAIN EBADF EFAULT EIDRM EINTR EINVAL EMSGSIZE ENOMEM ENOTSUP	EAGAIN EBADF EINVAL EMSGSIZE EINTR EINVAL ETIMEOUT	Receives a message from a message queue
msgctl	EACCES EPERM EFAULT EINVAL	EACCES EPERM EDIRM EFAULT EINVAL	Provides message control operations
msgget	EACCES ENOENT EEXIST ENOSPC	ENOMEM EACCES ENOENT EEXIST ENOSPC	Gets a message queue identifier
msgrcv	E2BIG EACCES ENOMSG EFAULT EINTR EIDRM EINVAL	E2BIG EACCES ENOMSG EFAULT EINTR EIDRM EINVAL	Reads a message from a queue
msgsnd	EAGAIN EACCES EFAULT EIDRM EINTR EINVAL ENOMEM	EAGAIN EACCES EFAULT EIDRM EINTR EINVAL ENOMEM	Sends a message

The default maximum size of a message queue (msg_qbytes) on Linux is set to the system parameter MSGMNB (16384 bytes). The msg_qbytes value can be raised beyond MSGMNB by using the msgctl function with the appropriate privileges. On Linux, the maximum size for a message text is set to MSGMAX (8192 bytes).

5.15.4. Shared Memory

On most UNIX systems, including Linux, the system administrator can edit a master file in the /etc directory to define limits for IPC mechanisms such as shared memory. AIX uses a different method. In AIX, upper limits are defined for IPC mechanisms, which are not configurable. The individual IPC data structures are allocated and deallocated as needed, so memory requirements depend on the current system usage of IPC mechanisms.

In AIX, the maximum size of a shared memory segment for 64-bit processes is 32TB. A 32-bit process cannot attach a shared memory segment larger than 2GB. 32-bit applications can use the shmat capability to obtain more than 11 segments when using the very large address space model without having to use extended shmat.

In Linux, the default maximum shared memory size is 32MB. For applications that require large address space, this value is often too small. To allow for applications to allocate memory above the 32MB mark (for instance, 512MB), the Linux systems administrator must take the steps listed in the following example (which sets the shared memory maximum to 512MB):

1.	Edit the file /etc/sysctl.conf.
2.	Set the shared memory size to 512MB (or higher) by adding the two lines listed here to the end of the file: kernel.shmmax = 536870912 kernel.shmall = 536870912
3.	Run the following command from the UNIX prompt: /sbin/sysctl -p

Every time the machine reboots, the shared memory will be set to this value. A more permanent solution is to change the value of SHMMAX in /usr/src/linux/include/asm/shmparam.h and rebuild your kernel:

The maximum value of SHMMAX can be set to 4GB-1 

Both AIX and Linux define the standard shmid_ds and ipc_perm structures, and member variable names are the same; however, the Linux version does not contain all the members that AIX has. The following member variables are not defined in the Linux shmid_ds structure:

__shm_cnattch (shm_cnattch[21]) /* In memory number attachments */ shm_handle   /* segment identifier */ __shm_extshm (shm_extshm21) /* page granularity shmat */ 

^[21] If_ALL_SOURCE is defined in AIX.

There are two different shared memory types available for most flavors of UNIXSystem V IPC and BSD mmap. AIX and Linux implement both. The functions provide a rich set of APIs that enable two or more processes to share access to a single chunk of memory. Table 5-13 shows there is no difference between AIX and Linux in their implementation of shared memory.

Table 5-13. AIX and Linux IPC API Comparison: Shared Memory

AIX	Linux	Description
shmat	shmat	Attach shared memory to data segment
shmctl	shmctl	Shared memory control operations
shmdt	shmdt	Detach shared memory from data segment
shmget	shmget	Get shared memory segment
shmop	shmop	Shared memory operations
mmap, mmap64	mmap, mmap64	Maps a file system object into virtual memory; mmap should be used on 64-bit machines
munmap	munmap	Unmaps pages of memory

Without IPC programming interfaces, processes and computers cannot share information across the network. A number of platform-independent APIsincluding Common Object Request Broker Architecture (CORBA), Distributed Computing Environment (DCE), and Message Bus (MBUS)use IPC calls to implement their services.