Linux Development Concepts

---

In this section, we are going to discuss technical terms, for the benefit of the Windows programmer. However, the section is devoted to introductory level definitions to familiarize you with the development process on the Linux platform. While defining these terms, we will attempt to compare them with similar concepts of the Windows operating system wherever applicable.

The File Systems in Linux

An operating system such as Linux permits us to store the information in a structured way. The information is stored in files, and files are stored within directories, and directories are stored within higher-level directories, and so on. In a very simple definition, a file system defines this (hierarchical) structure of files and directories and how they are organized. A file system design (by the operating system designers) is characterized by parameters such as access speeds, recovery of lost files, better utilization of the space on the hard disk, and so on. The Linux operating system supports a number of file systems, among which the most commonly known ones are ext2, ext3, minix, msdos, proc, swap, iso9660, hpfs, ntfs, and so on. Most of us are familiar with the file systems such as FAT16, FAT32, and NTFS defined in the Microsoft Windows operating systems, used to format the disk partitions such as C:, D:, E:, and so on.

In the beginning of the chapter, we discussed the concepts of hard disk partitions and looked at ways in which Linux and Windows could share hard disk space on the computer. A partition is nothing more than a logical allocation of the physical hard disk space or specific cylinders (and associated sectors) as a group. There are several advantages to making disk partitions. Each partition may be laid out with a different file system, if you need to. If one partition has too many bad sectors, that partition may be isolated, and if one particular partition fails to be mounted, you can decide not to use that partition (by deleting the appropriate entries from the /etc/fstab file) and continue booting your system with other partitions. This means that the problem can be isolated from the rest of the hard disk without causing major damage to the operation of the system. Therefore, any Enterprise-class system (or even your home-based Linux system) is typically partitioned. You will end up with at least two partitions on your Linux system one for the installation of the operating system and other packages and the other for the swap space. We will discuss swap space in more detail later in this chapter.

Once we understand the reasons for partitioning a hard disk, then we need to understand formatting. Theoretically, formatting is the process of implementing the file system on the partition, by which the operating system knows how the files are laid out and what the block size is that is used while writing and reading the files. On a Windows system, this is done by the format command. Different file systems implement this in different ways. Among the many file systems supported by Linux, the ext2 file system (also called the second extended file system) is very commonly used on most Linux systems. Most current kernel versions (as in Red Hat Linux 8.0/9.0) support the new ext3 file system, which is also called the third extended file system. File systems can be created by the mkfs command (mkfs stands for make file system). The syntax of this command in its simplest form is given here.

[root@etslinux root] mkfs –t <file system type> <partition> [root@etslinux root] mkfs –t ext2 /dev/hdc2

The first line gives the general form of the command, while execution of the command as shown in the second line creates an ext2 file system on the /dev/hdc2 partition. Like many UNIX/Linux commands, there are other options that may be used with the command as needed. The swap file system is created during installation when you choose a partition as swap space. It is important to note that a partition identified as swap partition cannot be shared for other purposes and the kernel takes full control of it after booting (and enabling the swap partition).

A file system needs to be mounted before attempting to access it. Usually, the file systems are mounted during booting and unmounted while shutting down the system. Mounting is the process of attaching the file system (on a specific partition) to a particular location within the big directory hierarchy, which starts as root, or / or associating with a subdirectory under the root directory. Please do not confuse the root directory with the root user account. They are different. The root directory is so called because it is the starting point of the directory hierarchy. In Linux, you do not see the different disk partitions with different letter designations such as C:, D:, and so on. Rather, they are identified as subdirectories under the root directory. During the time of installation, you have the complete choice of identifying which directories should act as mount points for which file systems. Theoretically, any directory or subdirectory in Linux can act as a mount point for a separate file system. The beauty of the architecture is that if you do not mount a separate file system for any directory, it is automatically considered part of the root file system, without a need of manual intervention. Table 2.1 shows a typical directory hierarchy in a Linux system.

Table 2.1: Typical Directory Hierarchy in Linux

Directory/Subdirectory	Description
/	Root directory, also called the root file system. If any of its sub-directories do not act as mount points for separate file systems, then contents of that subdirectory are stored in the root file system.
/boot	The /boot directory is used to store the boot image(s) of the kernel and usually requires much less disk space. You may create a separate file system under this mount point if you have space constraints on the primary hard drive and you are attempting to share the primary hard drive with the Windows operating system. Older versions of BIOS programs did not recognize operating systems installed beyond the first 1,024 cylinders of the primary hard drive. With the newer motherboards designed with newer BIOS, this limitation is relaxed, and an attempt to install the recent versions of kernels does not really pose any restrictions to installing the Linux operating system on any other partition. Therefore, you do not have to create a separate file system under this mount point.
/home	Every user account will have a home directory within this file system. For example, the home directory of user satya is created as /home/satya. Within the home directory, the user’s login profile, and profiles for any applications are stored. Individual application profiles are stored in further subdirectories; for example, the /home/satya/.kde directory is created and used for storing profiles related to KDE desktop. You may create a separate file system for /home if you created an exclusive partition during installation. If not, these directories are automatically created under the root directory within the root file system. The home directory of the root user account is created under the root directory within the root file system as /root and is automatically available when the booting process mounts the root file system. Because the root account is the administrator, it should be available when the system is booted even if there is a problem with mounting other file systems.
/bin/sbin	These directories store binary files (or executables and shared libraries) related to the operating system or other applications. These are part of the root file system. The /sbin directory usually contains files used during the boot process, while the /bin directory usually contains other administrative and user-level commands and utilities.
/lib	The /lib directory is used to store the libraries (and sometimes shared libraries) that are needed by the executable binary programs located in the /bin and /sbin directories. Often, custom user applications also make use of these libraries.
/dev	This directory contains file system entries that represent the devices that are attached to your Linux system and are necessary for the system to function properly. They should never be manually deleted. This directory forms part of the root file system.
/usr	This is the directory usually planned for housing the user applications, which means all the applications that do not become part of the operating system. The majority of the application software is usually installed into this directory and its subdirectories such as /usr/bin, /usr/sbin, /usr/lib, and so on. In a typical Linux system, this directory usually requires huge disk space and, hence, may act as a mount point for a separate file system.
/usr/local	The /usr/local directory is used to store any user-level and third-party applications. This directory will have subdirectories such as bin to store binary files, lib to store libraries, and so on.
/opt	The /opt directory (stands for optional) is considered to act as a mount point for some third-party applications. For example, when an application is developed and packaged, the designers of the application may determine that it should be installed in the /opt directory or /usr directory or any other directory. This directory also may act as a mount point for a separate file system, and this is always recommended if you have an available partition.
/tmp	This directory is used by the system and applications as temporary storage in which to store files for later deletion (as a working storage area). Usually, large Enterprises create a separate file system under this mount point, but it is not required for a personal Linux system.
/var	This directory is used to store variable data files such as spool directories and files, and files and directories meeting administrative and logging requirements. It is also used by some
/var	applications to store application-level data. However, it is the choice of the specific application whether to use this directory for storing its files. For example, an FTP server application may use subdirectories created under this directory to provide initial login location to the FTP client users. Enterprise-level systems may have a separate file system mounted at this mount point, but it is not required for a personal Linux system.
/proc	This directory is used by the system for maintaining information of running processes and current state of the kernel and acts as mount point for the proc file system. For convenience files of related entities (such as process or hardware category) may be grouped into individual virtual subdirectories under the /proc directory structure. Some programs such as ps, free, and netsta rely on the proc file system to access the kernel and process related information. The information in this file system is generated on the fly when you attempt to query on the contents of the directory. In other words, the /proc directory stores all virtual files. This is usually not considered to be a file system as it does not physically store anything.
/mnt or/media	This directory contains further subdirectories such as /mnt/cdrom, /mnt/floppy, /mnt/zip to mount a CD-ROM, floppy, or zip disk, respectively, whenever you want to access them. Although the /mnt-based mount points are used by Red Hat Linux, /media-based mount points are used by SuSE Linux while automounting these devices.

The /etc/fstab file stores the information about file systems that are mountable during booting. Administrators usually add new entries to and delete entries from this file if there are changes in the file systems. Sample contents of this file are shown here for your review.

# contents of /etc/fstab #<device>     <mountpoint>  <fstype> <options>       <dump> <fsck      order> /dev/hda1     /             ext3     defaults        1      1          /dev/hda4     /opt          ext3     defaults        1      2     /dev/hda3     /usr          ext3     defaults        1      2     /dev/hda7     swap          swap     defaults        0      0     /proc         /proc         proc     defaults        0      0     none          /dev/shm      tmpfs    defaults        0      0     /dev/cdrom    /mnt/cdrom    iso9660  noauto,user,exec,ro 0  0 /dev/fd0      /mnt/floppy   auto     noauto,owner,kudzu 0   0

The first four options are self-explanatory. A value in the <dump> column is used by the dump command to determine whether the file system needs to be backed up. A value of 1 indicates that the file system needs to be backed up. A value of 0 or no value in this column makes the dump command assume that the file system does not need to be backed up. A value for the last field indicates the order in which the fsck command should check the file system. The root file system should be checked first and hence should always have a value of 1. A value of 0 indicates that the file system does not have to be checked, which is usually the case with proc file system, CD-ROM, floppy drive, and the swap partition. The fsck command is used to check and optionally repair file systems, and is automatically run at boot time on all the file systems.

Please note that erroneous entries in this file could cause a partition fail to mount during booting. However, you can always manually mount a file system even after booting.

Syntax and usage of the mount command is given here:

[root@etslinux root] mount -t <file system type> <mount device> <mount           point>     [root@etslinux root] mount -t iso9660 /dev/cdrom /mnt/cdrom

The general syntax and the example are self-explanatory. Once the mount command is successful, the contents of the physical device are accessible through the directory specified as the mount point. As you notice from the example, the cdrom is mounted as iso9660 file system. If you need to mount a floppy, you may specify the msdos or ext2 file system as desired. The hard disks (and disk partitions) are usually mounted as ext2 or ext3, based on the type of file system created on the partition. If you attempt to mount a file system to a device that is already mounted, you will get a message indicating that the device is busy. Therefore, a device that is already mounted to a mount point should be unmounted before you make the next mount attempt. As you would expect, the command to unmount a device is umount, as shown in the following example:

[root@etslinux root] umount /dev/cdrom 

After successful unmounting of the device from the mount point, you cannot access its contents. The most recent versions of Linux, such as Red Hat 8.0 and SuSE 8.1, do automounting of CD-ROM/DVD/Floppy drive, if the corresponding driver has been installed. You will see an icon on the desktop for the mounted device. To unmount, you can click the right mouse button while keeping the mouse pointer on the icon and choose the unmount option from the displayed menu. You can do the same thing and choose the mount option in the menu to remount the device. Therefore, with the recent releases of Linux, you usually do not have to execute the manual mount and umount commands.

There are other options which may be specified with the mount command by using the –o <options> switch, such as the file system may be mounted read-only by specifying ro option, or you can put a restriction that no execution of binary files is permitted on mounting the file system by specifying noexec option, and so on.

Directories, Files, Links, and Devices

Once you understand the file systems and how they are mounted and accessed, the next step is to understand the concepts related to files, their types, and access permissions. Anything that you store on the computer is a file. In simple terms, a file stores a piece of information. The information may be a source program, an intermediate object file, or a binary executable file, or just a piece of data used by another program. The UNIX/Linux definition of a file is extended beyond this point, to include directories, links to other files, and even device files that represent specific hardware devices.

A simple file is a sequence of bytes, read and written in any way, as required by the application that accesses them. Examples are many, including a simple text file, a program source file, a graphic file, an executable binary file, and so on. Directories are a different type of files, which contain a number of files and subdirectories within themselves. Even an empty directory would contain two entries at the minimum, the “.” and the “..”. The “.” indicates the current directory, and the “..” indicates its parent directory. The peripheral (I/O) devices are identified and represented by special device files in Linux. These files are defined in the /dev directory. Examples include /dev/fd0, which is the device file for the first floppy drive. If you have a second floppy drive, it will be /dev/fd1. Similarly, /dev/hda, /dev/hdb, and /dev/hdc indicate first, second, and third hard drive in the system. These special files are associated with two device numbers, the major device number and the minor device number. The major device number represents the type of device (such as the floppy drive or the primary master IDE hard drive, or SCSI hard drive and so on), and the minor device number represents a particular device of the type identified by the same major device number. For example, if the major device number 3 identifies the primary master IDE hard drive on a particular Linux system, then there may be as many as 20 logical partitions of the drive identified by the minor device number values ranging from 0 through 19. This means the first partition on this drive is represented by the device /dev/hda0 with device numbers (3,0) and the tenth partition on the same drive is represented by the device /dev/hda9 with device numbers (3,9). The device files are also classified as block-device files and character-device files based on whether the device I/O is at block level or character level.

Every file in the file system is associated with an index known as inumber or inode. Each inode is linked to an index within a list of inodes called the ilist, which is a table stored within the file system. When the file system is created, the ilist with the maximum number of inodes is created. Every time a file is physically created, it is associated with one of the inodes in the list. The inode is a data structure containing the complete description of the file with details such as the user and the group ids owning the file, the file access permissions, the physical disk address where the file contents are stored, the size, a tag indicating the file type (such as directory or regular file, or special file), and so on. A directory contains a list of data structures. Each such data structure might represent a file and contains the name of the file and the inode of the file.

A separate ilist is maintained within each file system and is accessible when the file system is mounted. It is likely that the same inode may be in use in different file systems. But because each file system is associated with a different device, the combination of the (major and minor) device numbers and the inode together would uniquely identify a file (or directory) within the Linux system.

A physical file may be referenced by two different names, through the process of creating a hard link to the existing file. When a hard link is created with a new name, a file is created with the new name but with the same inode of the file for which the link is being created. Because the same inode value might exist across different file systems, it is not possible to create hard links across file systems (as the uniqueness of the file being linked is not guaranteed). For this reason, symbolic links are implemented. When a symbolic link is created to an existing file, a new file is created with the new file name that includes the path pointed to the file being linked. The (symbolic) link file will have its own inode and not the inode of the file being linked to. Because we are not using the same inode in the link file, it is possible to create symbolic links across file systems. Symbolic links are evaluated at runtime while the hard links are evaluated at the time of creating the links. In either case, the file contents are not duplicated.

The ln command may be used to create both the hard links as well as symbolic links, as shown in the examples here:

[root@etslinux root] ln [options].. <file to be linked> <link name> [root@etslinux root] ln orignalFile linkFile 

Execution of the command as shown in the second line creates a hard link with name linkFile that points to the file originalFile through the inode. Use the --symbolic option in the same command to create the symbolic link, as shown in the example here:

[root@etslinux root] ln --symbolic orignalFile linkFile

There are other options and variations of the command which would give more granulated results. Refer to the command manual pages for these details.

File Permissions

The next topic of interest that we should focus on is the file permissions. The permissions on the files are associated with three access categories—the read permission, write permission, and execute permission. When you view a directory listing using the ls command with the –l option, you can make a note of these permissions as r for read, w for write, and x for execute access, in the same order. You may set any combination of these values. If you do not set permission for one of these access categories, the corresponding value is displayed as a hyphen (-). For example, values displayed as r-x indicates that read and execute permissions are set while the write permission is not set, and values displayed as rw- indicate that read and write permissions are set while the execute permission is not set. In addition, every file can have three levels of access settings—the owner, group, and public. For each of the levels, a combination of the read, write, and execute permissions is specified. The first set of three values corresponds to the owner of the file, the second set of three values corresponds to the group that owns the file, and the third set of values corresponds to the general public (meaning everyone else). For example, a file permission setting of rwxr-xr-x indicates that the user who owns the file has all three permissions set; the group who owns the file and the general public have only read and execute permission on the file. Listing 2.4 demonstrates partial output of a directory listing command (output of ls –l command) for a normal directory, and Listing 2.5 shows a partial directory listing of /dev directory.

Listing 2.4: Directory listing for a normal directory

-rw-r--r--    1 root     root         3471 Oct  9 16:35 anaconda-ks.cfg drwx------    3 root     root         4096 Dec  3 07:23 Desktop drwx------    6 root     root         4096 Dec  1 18:50 evolution -rw-r--r--    1 root     root        24782 Oct  9 16:26 install.log -rw-r--r--    1 root     root         4096 Oct  9 16:22      install.log.syslog drwxr-xr-x    2 root     root         4096 Oct 23 21:42 jdev902home drwxr-xr-x    2 root     root         4096 Oct 23 21:42 jdev903home drwxr-xr-x    4 root     root         4096 Oct 23 21:47 jdevhome lrwxrwxrwx    1 root     root            8 Dec  5 20:20 linkP ->      Project1 -rw-r--r--    1 root     root            0 Dec  5 20:09 list.out -rw-r--r--    1 root     root         1356 Nov 30 16:25 lsmodout.txt drwx------    7 root     root         4096 Nov  2 17:45 Mail drwxr-xr-x    2 root     root         4096 Nov  5 06:13 PrintManPages -rwxr-xr-x    1 root     root       768440 Oct 18 07:27 Project1 -rw-r--r--    1 root     root       394348 Oct 18 07:27 Project1.o drwxr-xr-x    2 root     root         4096 Dec  5 20:09 screenshots -rw-r--r--    1 root     root          295 Oct 13 10:33 services.txt -rw-r--r--    1 root     root         3860 Nov  7 05:21 temp.txt -rw-r--r--    1 root     root         3632 Oct 15 07:40 Unit1.dcu -rw-r--r--    1 root     root       386388 Oct 18 07:27 Unit1.o -rw-r--r--    1 root     root        18631 Nov 13 07:23 xprinter.out

Listing 2.5: Partial directory listing of /dev directory

crw-------    1 root     root      10,  10 Aug 30 18:31 adbmouse crw-r--r--    1 root     root      10, 175 Aug 30 18:31 agpgart crw-------    1 root     root      10,   4 Aug 30 18:31 amigamouse crw-------    1 root     root      10,   7 Aug 30 18:31 amigamouse1 crw-------    1 root     root      10, 134 Aug 30 18:31 apm_bios drwxr-xr-x    2 root     root         4096 Oct  9 15:56 ataraid crw-------    1 root     root      10,   5 Aug 30 18:31 atarimouse crw-------    1 root     root      10,   3 Aug 30 18:31 atibm crw-------    1 root     root      10,   3 Aug 30 18:31 atimouse crw-------    1 root     root      14,   4 Aug 30 18:31 audio crw-------    1 root     root      14,  20 Aug 30 18:31 audio1 brw-rw----    1 root     floppy     2,   0 Aug 30 18:31 fd0 brw-rw----    1 root     floppy     2,   4 Aug 30 18:31 fd0d360 brw-rw----    1 root     floppy     2,  12 Aug 30 18:31 fd0D360 brw-rw----    1 root     floppy     2,  16 Aug 30 18:31 fd0D720 brw-rw----    1 root     floppy     2,  40 Aug 30 18:31 fd0h1440 brw-rw----    1 root     floppy     2,  28 Aug 30 18:31 fd0H1440 brw-rw----    1 root     disk       3,   0 Aug 30 18:31 hda brw-rw----    1 root     disk       3,   1 Aug 30 18:31 hda1 brw-rw----    1 root     disk       3,  10 Aug 30 18:31 hda10 brw-rw----    1 root     disk       3,  11 Aug 30 18:31 hda11 brw-rw----    1 root     disk       3,  12 Aug 30 18:31 hda12

]If you notice, the first entry has values for 10 column positions. The first position represents the type of file; a value of d indicates that it is a directory, a value of - is shown for regular files and hard links, a value of b is shown for block-device files, a value of c is shown for character-device files, a value of l is shown for symbolic links, and a value of s is shown for socket files. The next nine characters display the file permissions for the three levels: owner, group, and public. The second entry is a number. For directories, it shows the number of entries in the directory (which is at least 2 always) and a value 1 for all other type of files. The third entry is the user id of the owner of the file; the fourth entry is the group id that owns the file at group level. The entries that follow for all file types except special device files, are the size, last modified timestamp, and the name of the file. For the special device files the major and minor device numbers are displayed in place of size entry.

The file permissions are set using the chmod command. For this purpose, the three access levels—read, write, and execute—are collectively identified by an integer, whose binary equivalent (such as nnn) indicates which permissions are set or not set. For example, a value of 7 has the binary equivalent of 111. This means that all the three permissions (read, write, and execute) are set for the user category containing this value. Similarly, an integer value of 4 with its binary equivalent of 100 means that only the read permission is set, while the write and execute permissions are not set, and an integer value of 5 is equivalent to binary 101 with the meaning that read and execute permissions are set while write permission is not set. Finally, an integer value of 6 with its binary equivalent of 110 indicates that the read and write permissions are set and execute permission is not set. The chmod command uses the integer notation of permissions as shown below.

[root@etslinux root] chmod 755 <myprogram>

This example shows that the file myprogram has read, write, and execute permission for the owner and read and execute permission for the group and general public. The second form of the chmod command can specify how the current permissions on a file should be upgraded or reduced, as shown below

[root@etslinux root] chmod +x <myprogram> [root@etslinux root] chmod -w <myprogram>

The first line specifies to upgrade the current file permissions to add the execute access on the myprogram file, while the second line specifies to reduce the current file permission by removing the write access on the myprogram file.

Executables, Libraries, and Shared Objects

Now that we have gained basic knowledge about the files and files systems, we can turn our attention toward understanding the basic building blocks of application development. It is important to keep in mind that Linux provides a very efficient application development platform and is equipped with a number of GUI (graphical user interface)-based IDEs (Integrated Development Environments) in addition to the traditional command-line tools. You can write your programs in a programming language of your choice, such as C, C++, Delphi (Object Pascal), Java, to name a few. The vast majority of the traditional programming languages that are supported on UNIX are also available on Linux, although they are not mentioned in this book. However, the book will focus on the modern development platforms, particularly those that are designed based on the object-oriented programming paradigm. The scripting languages and programming languates will be discussed in Chapter 4, Shell Scripting Explored, and Chapter 5, Object-Oriented Programming, respectively, while we discuss here only the necessary building blocks from the operating system point of view.

The source program is usually written in one of the higher-level languages, and is compiled into an intermediate object file. This is done by the language compiler; for example, a C++ program could be compiled by the GNU C++ compiler, the C++ compiler that is shipped with Kylix3 from Borland Software Corporation, or another compiler. The object files generated by the compiler are not executable for several reasons. They need to be link-edited by the linker (or link-editor) from the GNU’s programming tools, from the Kylix3 command-line tools, or another link-editor supplied by a vendor, depending on how the intermediate object file was generated, and an application is typically comprised of multiple object files linked together to form an executable file. The most recent and standard executable format used in the Linux system is called ELF, which stands for Executable and Linking Format, which was originally developed and published by the UNIX System Laboratories as part of the Application Binary Interface. Later, this format was identified by several operating system architects as a standard portable object file format, which could streamline software development with a set of binary interface definitions extending across multiple operating environments. On the Linux system, the output of a link-edit step is typically an executable file in the ELF format. However, the Java program executables are in a different format called byte codes and are discussed in a later chapter.

Binary executable programs are primarily of three types; standalone executables, statically linked library files, and dynamically loadable shared object files. This concept is very similar to the Windows and other UNIX-clone operating systems. With the statically linked libraries, a copy of the called module’s code (from the linked library) is added to the executable program, thus increasing the size of the executable file. Whereas, while using the dynamically loadable shared objects, only a reference to the called module (from the shared object file) is maintained in the executable file, and the shared object file is loaded dynamically in the same process address space of the calling executable file during runtime. This process ensures that the executable file size does not increase vastly as the project keeps growing.

The make Utility

Large applications are composed of several modules (often developed in different programming languages) inter-dependent among themselves. When one or more modules are changed, it is often necessary to rebuild the entire application. The project makefile is a textual description (following a syntactical discipline) of the interdependencies of modules. The make utility is a command-line tool that rebuilds the entire application using the make file contents. The concept of using a make utility to build complex projects is not unique to Linux. It has been in existence since the early days of UNIX development, and is even implemented by Windows-based development tools such as Microsoft Visual C++, Borland C++ Builder, and so on. However, there are differences among these different platforms in the syntax used by the make command-line utility. Also, the newer editions of these development platforms introduced concepts such as Project and Project Groups, which use a more sophisticated (and easy to understand) XML-based project description files. These new utilities are undoubtedly very powerful, but it is important to remember that they are built upon the robust foundation laid by the make utility. Veteran UNIX/Linux programmers are still comfortable using the make utility, and therefore the GUI-based IDEs are providing tools to convert the XML-based project description files to the make file format.

Desktop versus Server Applications

Although most Windows programmers are familiar with a concept known as desktop applications, many UNIX-based programmers are not. This is because UNIX has been traditionally used as a server operating system, hosting several server-based applications providing services to the respective client applications. The Windows operating system has established itself almost as the de-facto desktop operating system, due to the large population of users in the world.

However, the Linux operating system is quite different from UNIX in one aspect; since the inception of the initial Linux releases, the developer community has focused its attention on making it a perfect desktop environment, at the same time inheriting the entire server-level capabilities of the UNIX architecture. Imagine having the very powerful UNIX environment working as a desktop operating system on your office desktop (or even your home-based entertainment system). That is exactly what gives you today’s Linux. In this sense, Linux may be viewed as UNIX++.

Because Linux is as powerful a desktop as a server, and because Linux is as powerful a desktop as Microsoft Windows, desktop application development is a necessity for Linux developers. Similarly, because Linux can be viewed as UNIX++, server-type application development is also a necessity for Linux developers. Therefore, the rest of the book is going to discuss the tools and techniques provided by the open source community as well as commercial vendors in developing desktop and server type applications.

Process Execution and Forking

When a program is executed in the Linux environment, a process is said to have been started (either by the init kernel thread or by another process). Without being in action, a program is a passive entity. Therefore, a process may also be viewed as a program in action, and it constitutes a set of machine instructions, data, and stack, competing for the system resources along with other processes. In other words, a process may also be considered as the execution context of a program. In a booted session of a Linux system, a number of processes are executing (even though you have not started any processes apparently). The init kernel thread itself is a process and is an instance of the /sbin/init program that was discussed in the beginning of the chapter. Every daemon started during the booting process is a process. Every window opened (including the command prompt window) constitutes a process. However, some processes may be short-lived, such as commands executed at the command prompt, which give an immediate output and exit. The init kernel thread and daemon processes are long-lived and exit only when the system is shutdown. It is important to understand that there may be many processes that appear to be executing simultaneously, as the kernel keeps scheduling them and when the new programs are invoked for execution. It is also likely that several processes may be separate instances of the same program. Unless the system has multiple processors, only one process runs at a time during its timeslice; this technique is called the multitasking or process scheduling. When one process is executing during its execution timeslice, the other processes would be waiting either for resources or for their turn to come. Every process is identified by a process id. The init kernel thread has the process id of 1 and is the ancestor of all the processes running in the system.

A process is initiated when its parent process makes the fork or clone system call. After the fork or clone system call is made, a clone of the parent process is created, and the new child process replaces the binary image of the parent process in the address space of the cloned parent process. Thus, the method of creating a child process by the fork system call is called forking. If we make the clone system call, the method may be termed as cloning. However, clone system call is unique to the Linux operating system, while fork is common across different UNIX environments.

Multi-threaded Applications and POSIX Threads

The concept of creating processes through the fork and clone system calls introduced a hierarchical (parent/child) relationship of a family of processes, as has been discussed in the previous topic. However, there are several situations that demand equality relationship between the different process threads created by the master thread (or the main program thread). In such cases, the POSIX (stands for Portable Operating System Interface) thread library is more useful than the fork and clone system calls. In addition, the other difference between the two concepts is that the multiple threads initiated by the same main program thread belong to the main program (and to the same process) and are all considered at par (even though one thread might have created another thread within the same program) and share the main program’s data structures; what is needed is a bit of thread synchronization. On the other hand, in the case of hierarchy of child processes created from a main process, each of the children is an independent process by itself (with a different process id) attached to its parent process, and communication between these processes needs a more complex IPC (stands for inter-process communication)-based approach, such as pipes, sockets, and message queues. The other reason that could favor multi-threaded programming over the multiprocess programming is the fact that creating a new process is more expensive and involves more overhead, such as interprocess communication, than creating a new thread. Windows programmers should already be familiar with multi-threaded programming, as Microsoft Windows operating systems also support the concept.

Interprocess Communication

By now, you should be familiar with the term process, and we noted earlier that multiprocess programming involves interprocess communication (IPC). As the term indicates, IPC is the mechanism by which two distinctly separate processes can communicate with each other and share the data across process address spaces. The different types of IPC mechanism supported on Linux are signals, pipes, sockets, semaphores, shared memory, and message queues. Although we make a mention of a couple of these IPC mechanisms, the other forms of IPC will be discussed in later chapters in more detail, with example programs.

Signals are used to signal (or trigger) asynchronous events to external processes. For example, shells may use signals to control the child processes, as in the kill command that could terminate a running process. The process may be designed to take appropriate action based on the signal that it receives from an external process. However, there are restrictions on which processes are permitted to send signals to other processes, particularly due to the permissions given to the owner of the process trying to send the signals. For example, the root super user can kill any process running on the system.

Piping is the mechanism of directing the output of one process to another. Also, pipes can be viewed as unidirectional byte streams connecting the standard output of one process to the standard input of another. Pipes are very effectively used in the Linux system even by the common users. When used in the command line, a pipe is identified by the | symbol. It is a common practice among the Linux (and UNIX) users to redirect the output of the ls –l command to the more command in order to facilitate page-wise viewing of the output of the first command, as shown here:

[root@etslinux root] ls –l | more

Because we execute this command at the shell prompt, the shell creates the temporary pipe between the two commands and facilitates the unidirectional communication between the two processes.

---