Processes

Before jumping into vulnerabilities that can occur based on a process's context and environment, you need to understand how processes operate in a typical UNIX system. A process is a data structure that an OS maintains to represent one instance of a program running in memory. A UNIX process has a considerable amount of state associated with it, including its own virtual memory layout and all the machine-specific information necessary to stop and start the flow of execution.

As noted in the previous chapter, each process has an associated process ID (PID), which is typically a small positive integer that uniquely identifies that process on the system. Most operating systems assign process IDs to new processes based on a systemwide counter that's incremented with each process that is created.

Note

Although this setup is typical, it's not universally true for all UNIX systems. One system that differs is OpenBSD, which selects a random PID for each new process. Generating random PIDs is intended to augment the security of an application that might use its PID in a security-sensitive context (such as using a PID as part of a filename). Using random PIDs can also make it more difficult for malicious parties to probe for the existence of running processes or infer other information about the system such as its current workload.

Process Creation

New processes are created in the UNIX environment with the fork() system call. When a process calls fork(), the kernel makes a nearly identical clone of that process. The new process will initially share the same memory, attributes, and resources as the old process. However, the new process will be given a different process ID, as well as some other minor differences; but in general, it's a replica of the original process.

When a new process is created with the fork() system call, the new process is referred to as a child of the original process. In UNIX, each process has a single parent process, which is usually the process that created it, and zero or more child processes. Processes can have multiple children, as they can make multiple copies of themselves with fork(). These parent and child relationships are tracked in the kernel structures that represent processes. A process can obtain the process ID of its parent process with the system call getppid(). If a process terminates while its children are still running, those child processes are assigned a "foster" parent: the special process init, which has a static PID (1) across all systems.

Consider what happens when a process calls fork(). The fork() system call creates another process that's a copy of the first one, and then the old and new processes are handed back over to the system to be scheduled at the next appropriate time. Both processes are running the same program, and both start processing at the instruction immediately after the system call to fork(). However, the return value of fork() differs based on whether the process is the parent or the child. The parent process receives the PID of the newly created child process, and the child process receives a return value of 0. A return value of -1 indicates that the fork() operation failed, and no child was spawned. Here's an example of creating a process with fork():

pid_t pid; switch (pid=fork()) {   case -1:     perror("fork");     exit(1);   case 0:     printf("I'm the child!\n");     do_child_stuff();     exit(1);   default:     printf("I'm the parent!\n");     printf("My kid is process number %d\n", pid);     break; }; /* parent code here */

If new processes are created only by the kernel duplicating an existing process, there's an obvious chicken-and-egg problem; how did the first process come about if no process existed beforehand to spawn it? However, there is a simple explanation. When a UNIX kernel first starts, it creates one or more special processes manually that help keep the system running smoothly. The first process is called init, and, as mentioned previously, it takes the special process ID of 1. init is synthesized from scratch when the kernel startsit is an Adam in the Garden of Eden, if you will. After that, userland processes are created with fork(). Therefore, almost every process can trace its origins back to a common ancestor, init, with the exception of a few special kernel processes.

fork() Variants

fork() is the primary way processes are created in a UNIX system. There are a few other similar system calls, but their use is generally deprecated or specific to a particular system. In older systems, vfork() was useful for creating a new process without having to suffer the performance hit of replicating its memory. It was typically used for the purpose of spawning a child process to immediately run a new program. As copy-on-write implementations of fork() became pervasive throughout UNIX, vfork() lost its usefulness and is now considered deprecated and bug prone. In some systems, a process created with vfork() has access to the virtual memory of its parent process, and the parent process is suspended from execution until the vfork() child runs a new program or terminates. On other systems, however, vfork() is just a wrapper for fork(), and address spaces aren't shared.

rfork() is another variation of fork() from the plan9 OS, although it isn't widely supported on other UNIX variants. It lets users specify the behavior of the forking operation at a more granular level. Using rfork(), a caller can toggle sharing process file descriptor tables, address spaces, and signal actions. clone() is a Linux variant of fork() that also allows callers to specify several parameters of the forking operation. Usually, these more granular process creation system calls are used to create threads, sometimes referred to as "lightweight processes." They enable you to create two or more processes that share a single virtual memory space, equivalent to multiple threads running in a single process.

Process Termination

Processes can terminate for a number of reasons. They can intentionally end their existence in several ways, including calling the library function exit() or returning out of their main function. These terminations result in the process calling an underlying exit() system call, which causes the kernel to terminate the process and release data structures and memory associated with it.

Certain signals can cause processes to terminate as well. The default handling behaviors for many signals is for the recipient process to be terminated. There's also a hard kill signal that can't be ignored or handled by a process. These kill signals can come from other processes or the kernel; a process can even send the signal to itself.

Any signal other than the kill and stop signals can be handled by your process, if you want. For example, if your program has a software bug that causes it to dereference a pointer to an unmapped address in memory, a hardware trap is generated that the kernel receives. The kernel then sends your process a signal indicating that a memory access violation has occurredUNIX calls this signal a "segmentation fault." Your process could handle this signal and keep on processing in light of this fault, but the default reaction is for the process to be terminated. There is also a library function abort(), which causes a process to send itself an abort signal, thus terminating the process. Signals are a complex topic area that is covered in depth in Chapter 13, "Synchronization and State."

fork() and Open Files

A child process is a nearly identical copy of its parent process, with only a few small differences. If everything is more or less identical, what happens to the files and resources the parent process already has open when it calls fork()? Intuition tells you that these open files must be available to both processes, which means the kernel must be handling sharing resources between the two processes. To understand this implicit file sharing relationship between a parent and a child, you need to be somewhat familiar with how resources are managed by the kernel on behalf of a process.

If you recall, you learned in Chapter 9 that when a process tells the kernel to open a file with the open() system call, the kernel first resolves the provided pathname to an inode by walking through all relevant directory entries. The kernel creates an inode data structure to track this file and asks the underlying file system to fill out that structure. The kernel then places an indirect reference to the inode structure in the process's file descriptor table, and the open() system call returns a file descriptor to the userland process that can be used to reference the file in future system calls.

System File Table

How the kernel places this "indirect" reference from the process file descriptor table to the inode structure hasn't been explained in much detail yet, but you explore this topic in depth in this section. Keep in mind that this chapter generalizes kernel internals across all UNIX implementations, so explanations capture the general behavior of the common UNIX process maintenance subsystem but it might not match a specific implementation exactly.

An open file is tracked by at least two different data structures, and each structure contains a different complementary set of data. The first of these structures is an inode structure, and it contains information about the file as it exists on the disk, including its owner and group, permission bits, and timestamps. The second structure, the open file structure, contains information about how the system is currently using that file, such as the current offset in the file for reading and writing, flags describing how the file is used (append mode, blocking mode, and synchronization), and the access mode specified when the file is first opened (read, write, or read/write). These open file structures (sometimes just called file structures) are maintained in a global table called the system file table, or the open system file table. This table is maintained by the kernel for the purposes of tracking all of the currently open files on the system.

Sharing Files

So what do these data structures have to do with fork()? Take a look at Figure 10-1, which shows the internal file data structures in a UNIX kernel after a fork(). Process 1000 has just forked a child process, process 1010. You can surmise that before the fork(), process 1000 had file descriptor 3 open to one of its configuration files. After the fork, you can see that the child process also has a file descriptor 3, which references the config file.

Figure 10-1. File data structures after fork

Both file descriptors point to the same open file structure, which tells you that the configuration file was opened with read/write access, and the current offset in the file is the location 0x1020. This open file structure points to the inode structure for the file, where you see that the file has an inode number of 0x456, has permission bits of octal 0644, and is owned by the bin user and bin group.

What does that tell you about how the kernel handles open files across a fork()? You can see that child processes automatically get a copy of the parent process's file descriptors, and one non-obvious result of this copying process is that both processes share the same open file structure in the kernel. So if you have a file descriptor open to a particular file, and you create a child process with fork(), your parent process can end up fighting with the child process if both processes try to work with that file. For example, if you're writing several pieces of data to the file in a loop, each time you write a piece, the file offset in the open file structure is increased past the piece you just wrote. If the child process attempts to read in this file from the beginning, it might do an lseek() on the file descriptor to set the file offset to the beginning of the file. If the child does this while you're in the middle of writing pieces of data, you start inadvertently writing data to the beginning of the file! Along those same lines, if the child changes the file to use a nonblocking interface, suddenly your system calls return with errors such as EAGAIN instead of blocking, as the parent process might expect.

As a code auditor, you need to be aware of resources that might be inadvertently available when a fork happens. Bugs involving leaked resources are often difficult to spot because descriptor sharing is an implicit operation the OS performs. Some basic techniques for recognizing vulnerabilities of this nature are described in the "File Descriptors" section later in this chapter.