11.6 Memory Requirements for Processes/Threads I hope that we now understand how scheduling can affect a process and its associated threads. This is not the only aspect of an application/process with which we need to concern ourselves . An application is not useful if it can't access data. The amount of data that the application requires is entirely application-dependent. In our discussions of swap space, we discussed how and when the operating system manages memory. In this section, we look at aspects of processes that use memory. Essentially, there are two types of data that a process will manipulate: user (private) data and shared data. User (private) data is unique to a process, while shared data is potentially accessible by other processes on the system. Two of the most commonly used shared objects are shared memory segments and shared libraries. Shared libraries contain code that is common to a number of programs. If shared libraries were not used, every program would need access to all the routines that it would ever reference. Commonly, this list of routines is similar from one program to the next . Using shared libraries reduces the amount of overall memory used in the system by cutting down the size of the user/private text portion of a process. Shared memory segments are created by a process using the shmget () system call. This will return a shared memory segment identifier pointing to the shared memory segment . They are commonly used by database applications to store large amounts of data in memory that will be accessible to multiple processes. One process will create the shared memory segment using a key , which acts like a filename. Subsequent processes can attach to the shared memory segment by supplying the same key . In some cases, a database startup routine may create all the shared memory segments for the entire application. Subsequent child processes will inherit all the shared memory segment identifiers created by its parent. In this way, child processes can attach to a shared memory segment in order to read and/or write to this shared data area. Processes can coordinate who can read and write to a shared memory segment by using a simple signaling concept known as semaphores. Semaphores are a way that processes can pass simple information between each other. This information is usually the simple value of a semaphore, and the value needs to be understood by the application. A simple example would be a semaphore encoding a stop/go value of 0 or 1. An application process could use the semaphore to indicate to another process that it was okay to write into memory by adjusting the value of the semaphore to 1. If the value of the semaphore were 0, the other process would know that it was not okay to write and would block wait for the semaphore to be changed by the original process. A set of semaphores is created by a process using the semget() system call. This returns an identifier that is used in a similar fashion to shared memory segment identifiers. There is a third common Inter-Process Communication (IPC) mechanism that allows processes to pass information between each other. The third mechanism is known as message queues . Message queues allow processes to pass small pieces (8KB maximum by default) of data between each other. The content of the message is not defined, so it is up to the application developer to decide what the content of the message will be. All three IPC mechanisms are shared objects. The largest of them is shared memory segments . Like other resources in the kernel, the number and size of IPC resources is limited by kernel parameters. The common IPC- related kernel parameters are listed in Table 11-4. Table 11-4. IPC-Related Kernel Parameters | Shared memory segments | Semaphores | Message queues | | shmmni : the maximum number of shared memory segments allowed in the system as a whole. | semmni : the maximum number of sets (identifiers) of semaphores allowed on the system as a whole. | msgmni : the maximum number of message queues allowed on the system as a whole. | | shmmax : the maximum size of an individual shared memory segment. | semmap : the size the free-space resource map used to locate free semaphore sets. | msgmax : the maximum size of an individual message. | | shmseg : the maximum number of shared memory segments to which an individual process can attach. | semmns : the maximum number of semaphores allowed per user. | msgmnb : the maximum size of all messages that can be queued simultaneously . | | | semmnu : the maximum number of undo operations pending on a semaphore. | msgseg : the number of individual message segments in a message queue. | | | semvmx : the maximum value that a semaphore is allowed to reach. | msgsssz : the size of a message segment. | | | semaem : the maximum amount that a semaphore value can be changed by an undo operation. | msgtql : the maximum number of messages allowed on the system in total. | | | | msgmap : the size of the free-space resource map used to locate free message queues. |
To view current usage of IPC resources, we use the ipcs command. The resources you wish to monitor will determine the options you use. For example, to look at the current shared memory segments in use, I can use the “m option.
root@hpeos003[] ipcs -mbop IPC status from /dev/kmem as of Sat Nov 22 03:06:46 2003 T ID KEY MODE OWNER GROUP NATTCH SEGSZ CPID LPID Shared Memory: m 0 0x411c28bb --rw-rw-rw- root root 0 348 667 667 m 1 0x4e0c0002 --rw-rw-rw- root root 1 61760 667 667 m 2 0x412001e0 --rw-rw-rw- root root 1 8192 667 679 m 3 0x301c5666 --rw-rw-rw- root root 1 1048576 1739 1739 m 2004 0x5e10001b --rw------- root root 1 512 1978 1978 m 205 0x00000000 D-rw------- root root 6 1052672 2060 2060 m 1006 0x00000000 D-rw------- www other 6 184324 2068 2068 m 6607 0x00001ed2 --rw-rw-rw- root sys 1 1048576 3569 3569 m 8 0x00001ed3 --rw-rw-rw- root sys 1 1048576 6204 6204 root@hpeos003[] root@hpeos003[] ps -fp 3569 -p 6204 0 UID PID PPID C STIME TTY TIME COMMAND root 3569 3022 232 02:59:43 pts/1 10:20 /finance/bin/finDB root 6204 3022 234 03:00:53 pts/1 9:19 /sales/bin/salesDB root@hpeos003[]
If we look at the last two shared memory segments, we can see that the CPID (Creator Process ID) that created the shared memory segment and the LPID (Last Process ID), the last process to attach to the shared memory segment, are the same. The NATTACH is the number of processes currently attached. It looks like these applications have just started up, created their shared memory segments, and attached to them, waiting for other processes to start up and attach to the shared memory segments. We can see the size of each of these segments ( SEGSZ ) is 1MB (1048576 bytes). Applications need to be coded in such a way that they trap and deal with signals appropriately. This includes trapping signals that will normally terminate a process. We need to ensure that the application behaves in such a way that before terminating, any previously used shared memory segments are removed from the system before the application actually terminates. If the application is not coded with this in mind, or if an administrator is a little careless with the kill command, we can have shared memory segments defined within the system with no one using them. If this situation is left unchecked, we could be in a situation where a new application starts up and attempts to create a new shared memory segment, but is refused because all the available shared memory has been used up. Unfortunately, some administrators think there is a command in UNIX called " kill “9 ". It never ceases to amaze some administrators that there are other signals you can send to a process that will cause the process to terminate. It appears to me that some administrators are just too impatient and want to get rid of a process the quickest and most sure-fire way they know how. A signal 9 certainly fits the bill. This can cause problems with shared memory segments. As mentioned above, if an application does not remove or is given the opportunity to remove a shared memory segment, it will be left allocated on the system whereby no other process can use that space for other shared objects. We need to be able to identify and remove such offending shared memory segments. The key here is to be extremely careful and look for shared memory segments with no attachments ( NATTACH =0) and no processes that created ( CPID ) or were attached to the segment previously ( LPID ). Take our finance application above. If we were to terminate the application with a signal 9, the application cannot trap the signal and it terminates immediately.
root@hpeos003[] kill 9 3569 root@hpeos003[] ps -fp 3569 UID PID PPID C STIME TTY TIME COMMAND root@hpeos003[]
This will leave the shared memory segment(s) for this application still allocated even though the application has terminated :
root@hpeos003[] ipcs -mbop IPC status from /dev/kmem as of Sat Nov 22 03:06:46 2003 T ID KEY MODE OWNER GROUP NATTCH SEGSZ CPID LPID Shared Memory: m 0 0x411c28bb --rw-rw-rw- root root 0 348 667 667 m 1 0x4e0c0002 --rw-rw-rw- root root 1 61760 667 667 m 2 0x412001e0 --rw-rw-rw- root root 1 8192 667 679 m 3 0x301c5666 --rw-rw-rw- root root 1 1048576 1739 1739 m 2004 0x5e10001b --rw------- root root 1 512 1978 1978 m 205 0x00000000 D-rw------- root root 6 1052672 2060 2060 m 1006 0x00000000 D-rw------- www other 6 184324 2068 2068 m 6607 0x00001ed2 --rw-rw-rw- root sys 0 1048576 3569 3569 m 8 0x00001ed3 --rw-rw-rw- root sys 1 1048576 6204 6204 root@hpeos003[]
The key fact here is that the NATTACH value is now zero. The CPID and LPID can be used to try to locate any processes that created or used the shared memory segment previously. If we cannot locate any such process and are confident that the application has shut down, we can remove this shared memory segment with the ipcrm command specifying the segment ID to identify the shared memory segment:
root@hpeos003[] ipcrm m 6607 root@hpeos003[] ipcs -mbop IPC status from /dev/kmem as of Sat Nov 22 03:06:46 2003 T ID KEY MODE OWNER GROUP NATTCH SEGSZ CPID LPID Shared Memory: m 0 0x411c28bb --rw-rw-rw- root root 0 348 667 667 m 1 0x4e0c0002 --rw-rw-rw- root root 1 61760 667 667 m 2 0x412001e0 --rw-rw-rw- root root 1 8192 667 679 m 3 0x301c5666 --rw-rw-rw- root root 1 1048576 1739 1739 m 2004 0x5e10001b --rw------- root root 1 512 1978 1978 m 205 0x00000000 D-rw------- root root 6 1052672 2060 2060 m 1006 0x00000000 D-rw------- www other 6 184324 2068 2068 m 8 0x00001ed3 --rw-rw-rw- root sys 1 1048576 6204 6204 root@hpeos003[]
The operating system will return that space to the pool of total available shared memory. 11.6.1 Locating private and shared data The operating system will locate particular types of user and shared data within particular address ranges in the process's Virtual Address Space. The address ranges equate to the idea of memory partitioning and memory quadrants . We have mentioned the concept of an address space and the fact that the operating system will partition the address space into quadrants. This idea is used to make it easier for the operating system to understand what type of data we are manipulating based on the address of the data object in question. Having an appreciation of how the address space is partitioned will explain some of the issues relating to managing memory at a process level. Figure 11-10 shows how 32-bit and 64-bit HP-UX performs memory partitioning. -
User text : Program instructions unique to the processes that are to be executed. -
User data : Includes Initialized data that corresponds to variables used by the User text. Uninitialized data (bss= b lock s tarted by s ymbol) are variables currently with no values assigned. Heap is undefined space used by the process through calls to malloc() and sbrk() . Memory Mapped Files allow applications to map files directly into the process's address space, bypassing access through the buffer cache (unbuffered IO). -
User stack : Used by the process when executing in User Mode to store variables, function arguments, and so on. -
UAREA : A structure maintained on a per-thread basis. The UAREA contains information that is swappable. Includes the current register context and system call information needed to run in kernel mode. -
Kernel stack : Used to execute instructions (system calls) when running in Kernel Mode. -
Shared libraries : Programs commonly use functions (and related data) that are the same program to program. -
Shared memory : A pool of memory that is addressable by all processes. Processes can use this space to communicate and share data. Figure 11-10. Address space layout. 
A 32-bit address space has a limitation of 4GB. These days, this is a concern for large applications, which commonly use large amounts of shared objects, principally shared memory segments. We discuss this shortly. User data, commonly referred to as private data, is unique to a process/thread. Private data includes program code as well as data elements defined in the program as well as data elements currently undefined but assigned space and values through calls to system calls such as malloc() . Even though each quadrant is theoretically 1GB in size, it is common to limit the size of these user/private areas by a series of kernel parameters: maxtsiz : the maximum size of a code/text segment maxdsiz : the maximum size of the user/private data segment maxssiz : the maximum size of the user stack segment There are corresponding 64-bit equivalent kernel parameters ( maxtsiz_64bit , and so on). The reason we limit these values is to ensure that a single process doesn't consume all the available memory. Remember, a process's view of memory is via its Virtual Address Space. A 32-bit process thinks it has access to 4GB of memory. A 64-bit process thinks it has access to 16EB of memory. This is the theoretical limit of a 64-bit operating system. HP-UX currently uses only 44 of the 64 bits. This limits the size of the entire address space to only 16TB. This should be enough for the foreseeable future. The fact that HP-UX uses the memory partitioning, known as quadrants, has an effect on how the address space is used. As we can see from Figure 11-10, certain types of data are stored at particular ranges of addresses with the address space. This allows HP-UX to know immediately that the process is trying to access user code, for example, because the virtual address used by the process falls within that particular address range. This is a useful feature for the operating system, but it does put limitations on individual processes. For 64-bit processes, the limitation is that each quadrant is only 4TB in size. This isn't really a limitation at present, so we won't bother ourselves with that. For 32-bit applications, even those running in a 64-bit operating system, the idea of quadrants puts significant limitations on the process. | 