Chapter 18: Other Application Development Topics
Download CD Content In This Chapter
-
Parsing Command-line Options with getopt and getopt_long
-
Time Conversion Functions
-
Gathering System Level Information with sysinfo
-
Mapping Physical Memory with mmap
-
Locking and Unlocking Memory Pages for Performance
Introduction
So far, we ve discussed a large number of topics relating to some of the more useful GNU/Linux service APIs. We ll now look at a number of miscellaneous core APIs that will complete our exploration of GNU/Linux application development. This will include the getopt function to parse command-line options, time and time conversion functions, physical memory mapping functions, and memory locking for high-performance applications.
| Note | The C language provides the means to pass command-line arguments into a program as it begins execution. The C main function may accept two arguments, argv and argc . The argc argument defines the number of arguments that were passed in, while argv is a character pointer array (vector), containing an element per argument. For example, argv[0 ] is a character pointer to the first argument (the program name ), and argv[argc-1] points to the last argument. |
Parsing Command-Line Options with getopt and getopt_long
The getopt function provides a simplified API for extracting command-line arguments and their options from the command line. Most arguments take the form
<application> -f <f-arg>
where -f is a command-line option and < f-arg > is the option for -f . Function getopt can also handle much more complex argument arrangements, as we ll see in this section.
The function prototype for the getopt function is provided as:
#include <unistd.h> int getopt (int argc, char * const argv[], const char *optstring); extern char *optarg; extern int optopt, optind;
The getopt function takes three arguments; the first two are the argc and argv arguments that are received through main . The third argument, optstring , represents our options specification. This consists of the options that we ll accept for our application. The option string has a special form. We define the characters that we ll accept as our options, and for each option that has an argument, we ll follow it with a : . Consider the following example option string:
"abc:d"
This will parse options such as -a , -b, -d, and also -c < arg >. We could also provide a double- colon , such as "abc::d" , which tells getopt that c uses an optional argument.
The getopt function returns an int that represents the character option. With this, three external variables are also provided as part of the getopt API. These are optarg , optopt , and optind . The optarg variable points to an option argument and is used to extract the option when one is expected. The optopt variable specifies the option that is currently being processed . The return value of getopt represents the variable. When getopt is finished parsing (returns -1 ), the optind variable represents the index of those arguments on the command line that were not parsed. For example, if a set of arguments are provided on the command line without any - options, then these arguments can be retrieved via the optind argument (we ll see an example of this shortly).
| Note | The application developer must ensure that all options are specified that are required for the application. The getopt function will provide the parsing aspect of command-line arguments, but the application must determine whether the options specified are accurate. |
Let s now look at an example of getopt that demonstrates the features that we ve touched upon (see Listing 18.1). At line 8 we call getopt to get the next option. Note that we call it until we get a -1 return, and it iterates through all of the options that are available. If getopt returns -1 , we exit our loop (to line 36).
At line 10, we start at switch construct to test the returns. If the 'h' character was returned (line 12 “14) we handled the Help option. At line 16, we handle the verbose option, which has an argument. Since we expect an integer argument after -v , we grab it using the optarg variable, passing it to atoi to convert it to an integer (line 17).
At line 20, we grab our -f argument (representing the filename). Since we re looking for a string argument, we can use optarg directly (line 21). At line 29, we test for any unrecognized options for which getopt will return '?' . We emit the actual option that was found with optopt (line 29).
Finally, at line 38 “42, we emit any options found that were not parsed using the optind variable. The getopt internally moves the nonoption arguments to the end of the argv argument list. Therefore, we can walk from optind to argc to find these.
 |
1 : #include <unistd.h> 2 : #include <stdio.h> 3 : 4 : int main(int argc, char *argv[]) 5 : { 6 : int c; 7 : 8 : while ((c = getopt (argc, argv, "hv:f:d")) != -1) { 9 : 10 : switch(c) { 11 : 12 : case 'h': 13 : printf("Help menu.\n"); 14 : break; 15 : 16 : case 'v': 17 : printf("Verbose level = %d\n", atoi(optarg)); 18 : break; 19 : 20 : case 'f': 21 : printf("Filename is = %s\n", optarg); 22 : break; 23 : 24 : case 'd': 25 : printf("Debug mode\n"); 26 : break; 27 : 28 : case '?': 29 : printf("Unrecognized option encountered -%c\n", optopt); 30 : 31 : default: 32 : exit(-1); 33 : 34 : } 35 : 36 : } 37 : 38 : for (c = optind ; c < argc ; c++) { 39 : 40 : printf("Non option %s\n", argv[c]); 41 : 42 : } 43 : 44 : 45 : /* 46 : * Option parsing complete... 47 : */ 48 : 49 : return 0; 50 : } | |
Many new applications support not only short option arguments (such as -a ) but also longer options (such as ”commmand=start ). The getopt_long function provides the application developer with the ability to parse both types of option arguments. The getopt_long function has the prototype:
#include <getopt.h< int getopt_long (int argc, char * const argv[], const char *optsring, const struct option *longopts, int *longindex);
The first three arguments ( argc , argv , and opstring ) mirror the getopt function. What differs for getopt_long are the final two arguments: longopts and longindex . The longopts argument is a structure that defines the set of long arguments that are desired to be parsed. This structure is defined as
struct option { const char *name; int has_arg; int *flag; int val; }; where name is the name of the long option (such as command ) and has_arg represents the argument that may follow the option ( for no argument, 1 for a required option, and 2 for an optional argument). The flag reference determines how the return value is provided. If flag is not NULL , the return value is provided by the fourth argument, val; otherwise the return value is returned by getopt_long .
Let s now look at an example of getopt_long . In this example, our application will accept the following arguments:
start stop command <command<
As we ll see, the getopt_long function is a perfect example of the use of a data structure to simplify the job of coding (see Listing 18.2).
The first item to note is that for the getopt_long function, we must include getopt.h (rather than unistd.h , as was done for getopt ). Our option data structure is defined at lines 4 “9. At line 5, we define the element for the ”start option. We define the name start , specify that it has no options, and then define the character that getopt_long will return once this option is found ('s') . The ”stop option is defined similarly (but return 't' on recognition). The ”command option identifies a required argument to follow (as defined by required_argument ) and will return 'c' when found on the command line.
At line 16 and 17, we see the call to getopt_long , which specifies our option string ( 'stc:' ) and our options structure ( longopts ). The return value, like function getopt , is -1 for no further options or a single character (as defined in the options structure).
When ”start is encountered, an 's' is returned and handled at lines 21 “23. The ”stop option is found, a 't' is returned and handled at lines 25 “27. When ”command is parsed, getopt_long returns 'c' , and we emit the command option at line 30 using the optarg variable. Finally, we identify unrecognized options at lines 33 “36 when '?' is returned from getopt_long (or an unknown option).
| |
1 : #include <stdio.h< 2 : #include <getopt.h< 3 : 4 : static struct option longopts[] = { 5 : { "start", no_argument, NULL, 's' }, 6 : { "stop", no_argument, NULL, 't' }, 7 : { "command", required_argument, NULL, 'c' }, 8 : { NULL, 0, NULL, 0 } 9 : }; 10 : 11 : 12 : int main(int argc, char *argv[]) 13 : { 14 : int c; 15 : 16 : while ((c = getopt_long (argc, argv, "stc:", 17 : longopts, NULL)) != -1) { 18 : 19 : switch(c) { 20 : 21 : case 's': 22 : printf("Start!\n"); 23 : break; 24 : 25 : case 't': 26 : printf("Stop!\n"); 27 : break; 28 : 29 : case 'c': 30 : printf("Command %s!\n", optarg); 31 : break; 32 : 33 : case '?': 34 : default: 35 : printf("Unknown option\n"); 36 : break; 37 : 38 : } 39 : 40 : } 41 : 42 : return 0; 43 : } | |
Any application that requires command-line configurability can benefit from getopt or getopt_long .
Time API
GNU/Linux provides a wide variety of functions to deal with time (as in time of day). Time is commonly represented by the tm structure, which is identified as:
struct tm { int tm_sec; /* seconds (0..59) */ int tm_min; /* minutes (0..59) */ int tm_hour; /* hours (0..23) */ int tm_mday; /* day of the month (1..31) */ int tm_mon; /* month (1..12) */ int tm_year; /* year (200x) */ int tm_wday; /* day of the week (0..6, 0 = Monday */ int tm_yday; /* day in the year (1..366) */ int tm_isdst; /* daylight savings time (0, 1, -1) */ }; A simplified representation is defined as the time_t structure, which simply represents the time in seconds (since the epoch 00:00:00 UTC, January 1, 1970). The time functions that we ll review in this section are these:
#include <time.h< time_t time (time_t *t); struct tm * localtime (const time_t *timep); struct tm * gmtime (const time_t *timep); char * asctime (const struct tm *tm); char * ctime (const time_t *timepDay); time_t mktime (struct tm *tm);
Grabbing the current time can be done with the time function, as:
time_t currentTime; currentTime = time (NULL);
Where NULL is passed to return the local time from the time function. The time can also be loaded into a variable by passing a time_t reference to the function:
time_t currentTime; (void) time (¤tTime);
With this time stored, we can now convert it into the tm structure using the localtime function. Putting it together with time , we get:
time_t currentTime; struct tm *tm_time; currentTime = time (NULL); tm_time = localtime (¤tTime); printf("%02d:%02d:%02d\n", tm_time-<tm_hour, tm_time-<tm_min, tm_time-<tm_sec); Converting time to an ASCII string is easily provided using the asctime or ctime function. The ctime function takes a time_t reference, while asctime takes a tm structure, as:
time_t currentTime; struct tm *tm_time; currentTime = time (NULL); printf("%s\n", ctime (¤tTime)); tm_time = localtime (¤tTime); printf("%s\n", asctime (tm_time)); The gmtime function breaks down a time_t variable into a tm structure, but in Coordinated Universal Time (CUT). This is the same as GMT (Greenwich Mean Time). The gmtime function is illustrated as:
tm_time = gmtime (¤tTime);
Finally, the mktime function converts the tm structure into the time_t format. It is demonstrated as:
tm_time = gmtime (¤tTime);
The entire set of functions is illustrated in the simple application shown in Listing 18.3.
| |
1 : #include <time.h< 2 : #include <stdio.h< 3 : 4 : int main() 5 : { 6 : time_t currentTime; 7 : struct tm *tm_time; 8 : 9 : currentTime = time (NULL); 10 : tm_time = localtime (¤tTime); 11 : 12 : printf("from localtime %02d:%02d:%02d\n", 13 : tm_time-<tm_hour, tm_time-<tm_min, tm_time-<tm_sec); 14 : 15 : printf("from ctime %s\n", ctime (¤tTime)); 16 : 17 : printf("from asctime/localtime %s\n", asctime (tm_time)); 18 : 19 : tm_time = gmtime (¤tTime); 20 : 21 : printf("from gmtime %02d:%02d:%02d\n", 22 : tm_time-<tm_hour, tm_time-<tm_min, tm_time-<tm_sec); 23 : 24 : printf("from asctime/gmtime %s\n", asctime (tm_time)); 25 : 26 : currentTime = mktime (tm_time); 27 : 28 : printf("from ctime/mktime %s\n", ctime (¤tTime)); 29 : 30 : return 0; 31 : } | |
Executing this application yields the following result:
$ ./time from localtime 22:53:02 from ctime Tue Jun 1 22:53:02 2004 from asctime/localtime Tue Jun 1 22:53:02 2004 from gmtime 04:53:02 from asctime/gmtime Wed Jun 2 04:53:02 2004 from ctime/mktime Wed Jun 2 05:53:02 2004 $
Gathering System Information with sysinfo
The sysinfo command allows an application to gather high-level information about a system, some of it very useful. The API for the sysinfo command is:
int sysinfo (struct sysinfo *info);
The sysinfo command returns zero on success and fills the info structure as defined by Table 18.1. Note that all sizes are provided in the units defined by mem_unit .
| Element | Description |
|---|---|
| uptime | The current uptime of this system in seconds |
| loads[0] | System load average for 1 minute |
| loads[1] | System load average for 5 minutes |
| loads[2] | System load average for 15 minutes |
| totalram | Total usable main memory |
| freeram | Available main memory |
| sharedram | Amount of memory that s shared |
| bufferram | Amount of memory used by buffers |
| totalswap | Total swap space |
| freeswap | Free swap space |
| procs | Number of currently active processes |
| totalhigh | Total amount of high memory |
| freehigh | Free amount of high memory |
| mem_unit | Memory unit size in bytes |
Gathering the system information is very simple, as illustrated in Listing 18.4. Note that uptime has been further decomposed to provide a more meaningful representation (line 16 “25).
| |
1 : #include <sys/sysinfo.h< 2 : #include <stdio.h< 3 : 4 : int main() 5 : { 6 : struct sysinfo info; 7 : int ret; 8 : int days, hours, minutes, seconds; 9 : 10 : ret = sysinfo (&info); 11 : 12 : if (ret == 0) { 13 : 14 : printf("Uptime is %ld\n", info.uptime); 15 : 16 : days = info.uptime / (24 * 60 * 60); 17 : info.uptime -= (days * (24 * 60 * 60)); 18 : hours = info.uptime / (60 * 60); 19 : info.uptime -= (hours * (60 * 60)); 20 : minutes = info.uptime / 60; 21 : info.uptime -= (minutes * 60); 22 : seconds = info.uptime; 23 : 24 : printf("Uptime %d Days %d Hours %d Minutes %d Seconds\n", 25 : days, hours, minutes, seconds); 26 : printf("One minute load average %ld\n", info.loads[0]); 27 : printf("Five minute load average %ld\n", info.loads[1]); 28 : printf("Fifteen minute load average %ld\n", info.loads[2]); 29 : printf("Total Ram Available %ld\n", info.totalram); 30 : printf("Free Ram Available %ld\n", info.freeram); 31 : printf("Shared Ram Available %ld\n", info.sharedram); 32 : printf("Buffer Ram Available %ld\n", info.bufferram); 33 : printf("Total Swap Size %ld\n", info.totalswap); 34 : printf("Available Swap Size %ld\n", info.freeswap); 35 : printf("Processes running: %d\n", info.procs); 36 : printf("Total high memory size %ld\n", info.totalhigh); 37 : printf("Available high memory %ld\n", info.freehigh); 38 : printf("Memory Unit size %d\n", info.mem_unit); 39 : } 40 : 41 : return 0; 42 : } | |
Much of this information can also be gathered from the /proc filesystem, but that is more difficult due to the parsing that s necessary. Some information is provided in /proc/uptime , /proc/meminfo , and /proc/loadavg .
| Note | The proc filesystem is a virtual filesystem that contains runtime information about the state of the operating system. The proc filesystem can be used to inquire about various features by ˜ cat ing files in the proc filesystem. For example, we could identify all processes in the system ( /proc/# ), information about the CPU ( /proc/ cpuinfo ), the devices found on the PCI buses ( /proc/pci ), the kernel modules currently loaded ( /proc/modules ), and much more runtime information. |
Mapping Memory with mmap
While not completely related to shared memory, the mmap API function provides the means to map file contents into user program space. The prototype function for mmap (and munmap to unmap the memory) is defined as:
#include <sys/mman.h> void * mmap (void *start, size_t length, int prot, int flags, int fd, off_t offset); int munmap (void *start, size_t length);
The mmap function takes in a file byte offset ( offset ) and tries to map it to the address defined by the caller ( start ) from the file descriptor fd (we ll look at what this means shortly). Commonly, the start address is defined as NULL , allowing mmap to map it to whatever local address it chooses. This address (local mapping) is returned by the mmap function. The length of the region is defined by length . The caller defines the desired memory protection through the prot argument, which can be PROT_EXEC (region may be executed), PROT_READ (region may be read), PROT_WRITE (region may be written), or PROT_NONE (pages can t be accessed). Combinations of protections can be defined. Finally, the type of mapped object is defined by the flags argument. This can be MAP_FIXED (fail if the start address can t be used for the local mapping), MAP_SHARED (share this mapping with other processes), or MAP_PRIVATE (create a private copy-on-write mapping). The caller must specify MAP_SHARED or MAP_PRIVATE .
| Note | The offset and start arguments must be on page boundaries. The length argument should be a multiple of the page size. |
GNU/Linux also provides some other nonstandard flags that are defined in Table 18.2.
| Flag | Use |
|---|---|
| MAP_NORESERVE | Used with MAP_PRIVATE to instruct the kernel not to reserve swap space for this region. |
| MAP_GROWSDOWN | Used by the kernel for stack allocation. |
| MAP_ANONYMOUS | This region is not backed by a file ( fd and offset arguments of mmap are therefore ignored). |
The munmap function simply unmaps the memory mapped by mmap . To munmap , the address returned by mmap is provided along with the length (which was also specified to mmap ).
Let s look at an example of mmap and munmap . In Listing 18.5, we find an application that maps physical memory and makes it available for read. This application first creates a file descriptor of the file /dev/mem , which represents the physical memory available.
| Note | It s important to note that /dev/mem should be used only for read operations. Writing can be dangerous to the point of crashing your system. Finally, it s also a large security hole, and therefore its use should be avoided. |
Our sample application allows the base address and length to be defined on the command line (lines 15 “32). At line 34, we open /dev/mem for read. The file /dev/mem permits access to physical memory space (for which we ll use mmap to map physical memory into the process s memory space). At lines 38 “40, we use mmap to map the requested region into the process s address space. The return value is the address from which the memory can be accessed. We then perform a loop to read and printf the addresses and their contents (lines 45 “57). We finally clean up by unmapping the memory with munmap (line 59) and then closing the /dev/mem file descriptor at line 67.
| |
1 : #include <stdio.h> 2 : #include <unistd.h> 3 : #include <fcntl.h> 4 : #include <stdlib.h> 5 : #include <errno.h> 6 : #include <sys/mman.h> 7 : 8 : int main(int argc, char *argv[]) 9 : { 10 : int fd; 11 : unsigned char *addr, *waddr, count; 12 : int length; 13 : off_t offset; 14 : 15 : if (argc < 3) { 16 : 17 : printf("Usage is phymap <address< <length<\n"); 18 : exit(-1); 19 : 20 : } 21 : 22 : if (argv[1][1] == 'x') { 23 : sscanf(argv[1], "0x%x", &offset); 24 : } else { 25 : sscanf(argv[1], "%d", &offset); 26 : } 27 : 28 : if (argv[2][1] == 'x') { 29 : sscanf(argv[2], "0x%x", &length); 30 : } else { 31 : sscanf(argv[2], "%d", &length); 32 : } 33 : 34 : fd = open("/dev/mem", O_RDONLY); 35 : 36 : if (fd != -1) { 37 : 38 : addr = (unsigned char *) mmap (NULL, length, 39 : PROT_READ, MAP_SHARED, 40 : fd, offset); 41 : 42 : 43 : if (addr != NULL) { 44 : 45 : waddr = addr; 46 : 47 : for (count = 0 ; count < length ; count++) { 48 : 49 : if ((count % 16) == 0) { 50 : 51 : printf("\n%8p : ", waddr); 52 : 53 : } 54 : 55 : printf("% 2x ", *waddr++); 56 : 57 : } 58 : 59 : munmap (addr, length); 60 : 61 : } else { 62 : 63 : printf("Unable to map memory.\n"); 64 : 65 : } 66 : 67 : close(fd); 68 : printf("\n"); 69 : 70 : } 71 : 72 : return 0; 73 : } | |
Using this application (let s call it phymap ) is illustrated below. Here we peek at the address 0x000c1000 for 64 bytes.
# ./phymap 0x000c1000 64 addr = 0x40017000 (Success) 0x40017000 : 56 57 a0 49 04 e8 c0 fe 32 ed 41 c1 e9 03 32 e4 0x40017010 : e8 0d 67 2e 8b b5 e6 08 2e 8a 44 0c d1 e8 f7 e1 0x40017020 : 5f 5e 07 5a 59 c3 33 c9 a4 fe c1 8a 2c 0a ed 75 0x40017030 : f7 fe c1 a4 32 ed c3 50 53 32 e4 b0 11 b3 80 e8 #
The address shown here is the local address that s been mapped in our address space. While it s different than our requested 0x000c1000 , the 0x40017000 represents the same region for our process.
Locking and Unlocking Memory
Let s look at an additional set of functions that are very useful to high-performance applications. The memory-locking functions permit a process to lock some or all of its storage that it is never swapped out of memory. The result is greater performance for the application, since it never has to suffer paging penalties in a busy system, but this does require the proper permissions.
The functions of interest for locking and unlocking memory are:
#include <sys/mman.h> int mlock (const void *addr, size_t len); int munlock (const void *addr, size_t len); int mlockall (int flags); int munlockall (void);
The addr and len arguments must be on page boundaries. Let s now look at the lock/unlock pairs of functions in detail.
The mlock function takes an address ( addr ) for which the memory pages that represent the region are to be locked. The len argument defines the size of the region to lock (meaning that one or more pages may be locked by the operation). A return value of zero means that the pages are locked. When the application is finished with the memory, a call to munlock makes the pages available for swapping. A return of zero represents success of the unlock system call.
The following code example illustrates locking a page of memory and then unlocking it (see Listing 18.6). Our buffer is a locally created array of characters.
| |
1 : #include <stdio.h> 2 : #include <sys/mman.h> 3 : 4 : char data[4096]; 5 : 6 : int main() 7 : { 8 : int ret; 9 : 10 : ret = mlock (&data, 1024); 11 : 12 : printf("mlock ret = %d\n", ret); 13 : 14 : ret = munlock (&data, 1024); 15 : 16 : printf("munlock ret = %d\n", ret); 17 : 18 : return 0; 19 : } | |
At line 10, we call mlock with a reference to our global buffer ( data ) and its length. We unlock this page by calling mlock with our buffer and size (identically to the call to mlock ). The entire page containing the buffer is locked, in the event it falls under the bounds of a page.
| Note | Child processes do not inherit memory locks (created by mlock or mlockall ). Therefore, if a region of memory is created and subprocesses also created to operate upon it, each child process should perform its own mlock . |
The mlockall API function locks all memory (disable paging) for a process s entire memory space. This includes not only the code and data for the process, but also its stack, shared libraries, shared memory, and other memory mapped files.
The mlockall function takes a single argument, which represents the scope of the lock to be provided. The user can specify MCL_CURRENT , which locks all pages that are currently mapped into the address space of the process, or MCL_FUTURE , which locks all pages that will be mapped into the address space of the process in the future. For example:
/* Lock currently mapped pages */ mlockall (MCL_CURRENT); /* Lock all future pages */ mlockall (MCL_FUTURE);
We can also define that all current and future pages are locked into memory by performing a bitwise OR the flags together, such as:
mlockall (MCL_CURRENT MCL_FUTURE);
If insufficient memory is available to lock the current set of pages, an ENOMEM error is returned. If MCL_FUTURE is used, and insufficient memory exists to lock a growing process stack, the kernel will throw a SIGSEGV (segment violation) signal for the process, causing it to terminate.
The munlockall system call reenables paging for the pages mapped for the calling process. It takes no arguments and returns zero on success:
/* Unlock the pages for this process */ munlockall ();
The munlockall system call should be called once the process has completed its real-time processing.
EAN: 2147483647
Pages: 203