18.10 MULTITHREADED PROGRAMMING IN CC | Programming with Objects: A Comparative Presentation of Object Oriented Programming with C++ and Java

18.10 MULTITHREADED PROGRAMMING IN C/C++

As was mentioned in the introduction to this chapter, one has to use an external thread package to do multithreaded programming in C++ since support for multithreading is not an integral part of the language. This is also true of C—that the language standard itself does not provide for multithreading. Over the years, various thread packages have been proposed for multithreading in C and, through C, for C++. Of these, the POSIX threads have emerged as the de facto standard.^[15] Since C++ thread packages are generally built on top of C thread packages, it is a good idea to first grasp the POSIX threads in C. That is what we will do in this section.

The main library for POSIX threads in C is pthreads.h. It defines a data type called pthread_t for threads. A new thread is created by a call to pthread_create, a function of four parameters. Its various arguments are set in the following manner:

The first argument is a pointer to an object of type pthread_t.
The second argument, of type pthread_attr_t, can beused to specify thread attributes, such as the stack size to use, scheduling priority, and so on. If set to NULL, the default values are used for all the thread attributes.
The third argument is a pointer to a function that is to be executed by the thread. This function must be of type (void* (*)(void*)) or of a type that can be cast to the specified type.
The final argument is a pointer to a data block that can be passed to the function specified in the third argument.

To illustrate:

     pthread_t thread1;                                              //(A)     char* message1 = "hello";                                       //(B)     pthread_create(&thread1,                                        //(C)                         NULL,                         (void* (*)(void*)) &print_message_function,                         (void*) message1);

Line (A) declares the identifier thread1 to be of type pthread_t—thread1 can serve as a convenient name for the thread. Line (B) declares a message that we would want the thread to print out. Line (C) then creates the thread. We assume that the function to be executed by the thread is named print_message_function.^[16]

In the above example, we used NULL for the second argument in the call to thread_create function. As was mentioned earlier, this sets all thread attribute values to their default values.^[17] In general, the operatingbehavior of a thread is controlled by an attribute object of type pthread_attr_t if specified as the second argument to pthread_create. To give the reader a sense of the richness of a POSIX thread, we will now list all the attributes one can specify for it.^[18]

detachstate

This attribute controls whether the thread is created in a joinable state, in which case the value of the attribute is PTHREAD_CREATE_JOINABLE, or in a detached state, in which case the value of the attribute is PTHREAD_CREATE_DETACHED. The former is the default value. If a thread is created in a detached state, thread coordination cannot be achieved by invoking a function like pthread_join on it.y^[19] Additionally, whereas a thread created in a detached state immediately frees up the resources occupied by it when the thread runs to completion, a joinable thread will not do so until pthread_join is called for it. For example, the thread ID (which is the first argument to pthread_create) and the exit status of a joinable thread are retained until some other thread calls pthread_join on it. (Therefore, it is imperative that pthread_join be called for each joinable thread to avoid memory leaks.) It is possible to dynamically change this attribute for a previously created joinable thread by calling pthread_detach(pthread_t threadID) on it.
schedpolicy

This attribute selects the scheduling policy to be used for the thread. Its value can be SCHED_OTHER for regular, nonrealtime scheduling, or SCHED_RR for realtime, round-robin, or SCHED_FIFO for realtime first-in first-out. The default value is SCHED_OTHER. The realtime policies are available only to processes that carry superuser privileges. The scheduling policy of a thread can be changed after the thread is created by invoking pthread_setschedpolicy.
schedparam

This sets the scheduling priority for the thread. It is only meaningful for realtime scheduling policies. The default value of this attribute is 0. The scheduling priority of a thread can be changed after the thread is created by invoking pthread_setschedparam.
inheritsched

This attribute indicates whether the scheduling policy and scheduling parameters for a newly created thread are to be determined by the values of the attributes schedpolicy and schedparam, or should they be inherited from the parent thread. The two values for this attribute are PTHREAD_EXPLICIT_SCHED and PTHREAD_INHERIT_SCHED. The former is the default.
scope

This attribute defines the scope to be used for resolving the scheduling contention. The issue here is whether the thread priority should be relative to all other processes competing for time on the processor, or relative to just the other threads in the same process. The POSIX standard specifies two values for this attribute: PTHREAD_SCOPE_SYSTEM and PTHREAD_SCOPE_PROCESS. The former is the default, which says that a thread must compete with all other processes for time on the processor. Linux only supports this default value.

The value of any attribute in the pthread_attr object can be set by invoking

     int pthread_attr_setattrname()                                  ^^^^^^^^

and retrieved by

     int pthread_attr_getattrname()                                  ^^^^^^^^

where the part of the function call that is underscored with small uparrows is to be replaced by the name of the attribute. For example, to set the scheduling policy,

     int pthread_attr_setschedpolicy(const pthread_attr_t* attr,                                                             int* policy)

and to retrieve the policy

     int pthread_attr_getschedpolicy(const pthread_attr_    t* attr,                                                          int*; policy)

In both cases, the successful completion of the function calls is indicated by a return value of 0. In the second case, if the policy is what is specified by the second argument, the return value will be 0.

We will now show a simple program for illustrating thread creation. This program does the same thing as our first Java multithreaded program, ThreadBasic.java, of Section 18.1 and, like the Java program, suffers from some serious flaws that we will address next. Since our intent here is to focus on the basics of thread creation, we will be content with the default values for the thread attributes.

 
 //ThreadBasic.c #include <pthread.h> #include <stdio.h> void* print_message(void*); main() {     pthread_t thread1, thread2, thread3;     pthread_attr_t attr1, attr2, attr3;     char* message1 = "Good";     char* message2 = " morning";     char* message3 = " to";     pthread_attr_init(&attr1);     pthread_attr_init(&attr2);     pthread_attr_init(&attr3);     pthread_create(&thread1, &attr1,         (void* (*)(void*)) &print_message, (void *) message1);     pthread_create(&thread2, &attr2,         (void * (*)(void*)) &print_message, (void *) message2);     pthread_create(&thread3, &attr3,         (void * (*)(void*)) &print_message, (void *) message3);     fprintf(stdout, "%s", " you!" );     exit(0);                                                            //(A) }     // Function to be executed in each thread:     void* print_message(void* ptr) {         char* message;         message = (char*) ptr;         // sleep(2);                                                    //(B)         fprintf(stdout, "%s", message);         fflush(stdout);                                                 //(C)         pthread_exit(0);                         // terminates thread     }

If this program is stored in a file called ThreadBasic.c, it can be compiled by

     gcc ThreadBasic.c -lpthread

One would think that the output of this program would be

     Good morning to you!

but what you actually get would depend on the outcome of the race condition that exists in the program. And if you uncomment line (B) of the print_message function, the program will not at all behave as intended.

As was the case with the Java program of Section 18.1, the problem with this program is that the parent thread or the process in which main is being executed is racing to get to exit(0) in line (A) in competition with the child threads. If the three child threads are not done with their work by the time the main parent thread gets to executing exit(0), the process will terminate, killing all the threads with it. Additionally, even if main did not complete its work before the child threads, there is no guarantee that the child threads would do their jobs in the order in which they are created. To accentuate the race between main and the three threads, you can uncomment line (B), which would cause the program to print out just you! and nothing else, as main would run to termination before the child threads can finish up.^[20]

The program shown above can be partially fixed by using pthread_join. The fix shown here, meant to illustrate the usage of pthread_join, eliminates the race condition between the parent thread or the process running main and the child threads. As was mentioned earlier in this section, a call to the function

     int pthread_join(pthread_t thread, void** status);

makes the calling thread wait until the thread named in the first argument terminates either by calling pthread_exit or by being canceled. It is not uncommon to call this function with NULL for the second argument, as in the following invocation

     pthread_join(thread, NULL);

if it is not desired to find out whether a thread terminated naturally by the invocation of pthread_exit or by being canceled. If the second argument to this function is not NULL, the value returned by the terminating thread is stored as a code in status. This value will either be the argument given to pthread_exit when it was invoked or the enumeration symbolic constant PTHREAD_CANCELED if the thread was canceled.

Here is the program:

 
 //ThreadBasicWithJoin.c #include <pthread.h> #include <stdio.h> void* print_message(void*); main() {     pthread_t thread1, thread2, thread3;     pthread_attr_t attr1, attr2, attr3;     char* message1 = "Good ";     char* message2 = " morning";     char* message3 = " to";     int status1;     int status2;     int status3;     pthread_attr_init(&attr1);     pthread_attr_init(&attr2);     pthread_attr_init(&attr3);     pthread_create(&thread1, &attr1,           (void* (*)(void*)) &print_message, (void *) message1);     pthread_create(&thread2, &attr2,           (void * (*)(void*)) &print_message, (void *) message2);     pthread_create(&thread3, &attr3,           (void * (*)(void*)) &print_message, (void *) message3);     pthread_join(thread1, (void*) &status1);     pthread_join(thread2, (void*) &status2);     pthread_join(thread3, (void*) &status3);     // optional code for processing status1, status2, status3     fprintf(stdout, "%s", " you!");     exit(0); }     // Function to be executed by each thread:     void* print_message(void* ptr) {         char* message;         message = (char*) ptr;         // sleep(2);         fprintf(stdout, "%s", message);         fflush(stdout);         pthread_exit(0);     }

18.10.1 Demonstrating Thread Interference with POSIX Threads

As was the case with Java multithreading, in addition to the race conditions, one also has to guard against thread interference. To demonstrate thread interference when using POSIX threads, we will show the following example that parallels the first Java example of Section 18.4 for illustrating thread interference.

As in the Java example UnsynchedSwaps.java, we launch four threads, each repeatedly modifying the two data members of a DataObject object, dobj, in such a way that the sum of the two data members should always add up to 100., of Section 18.1 and, like the Java program, suffers from some serious flaws that we will address next. Since our intent here is to focus on the basics of thread creation, we will be content with the default values for the thread attributes.

 
 //UnsynchedSwaps.c #include <pthread.h> #include <stdio.h> #include <time.h> typedef struct {     int dataItem1;     int dataItem2; } DataObject; //initialization allocates memory for dobj: DataObject dobj = { 50, 50 }; void keepBusy(double howLongInMillisec); void itemSwap(DataObject* dptr); void test(DataObject* dptr); void repeatedSwaps(DataObject* dptr); main() {     pthread_t t1, t2, t3, t4;     pthread_create(&t1, NULL, (void* (*)(void*)) repeatedSwaps, &dobj);     pthread_create(&t2, NULL, (void* (*)(void*)) repeatedSwaps, &dobj);     pthread_create(&t3, NULL, (void* (*)(void*)) repeatedSwaps, &dobj);     pthread_create(&t4, NULL, (void* (*)(void*)) repeatedSwaps, &dobj);     pthread_join(t1, NULL);     pthread_join(t2, NULL);     pthread_join(t3, NULL);     pthread_join(t4, NULL); } void keepBusy(double howLongInMillisec) {     int ticksPerSec = CLOCKS_PER_SEC;     int ticksPerMillisec = ticksPerSec / 1000;     clock_t ct = clock();     while (clock() < ct + howLongInMillisec * ticksPerMillisec)         ; } void itemSwap(DataObject* dptr) {     int x = (int) (-4.999999 + rand() % 10);     dptr->dataItem1 -= x;     keepBusy(10);     dptr->dataItem2 += x; } void test(DataObject* dptr) {     int sum = dptr->dataItem1 + dptr->dataItem2;     printf("%d\n", sum); } void repeatedSwaps(DataObject* dptr) {     int i = 0;     while (i < 20000) {         itemSwap(dptr);         if (i % 4000 == 0) test(dptr);         keepBusy(1); // in milliseconds         i++;     } }

The logic of the above program is the same as that of the Java program UnsynchedSwaps.java of Section 18.4; the reader is referred to the Java implementation for a more detailed explanation. The cause of thread interference also remains the same. That the threads are stepping on one another is demonstrated by the following output, which is similar to what we obtained with UnsynchedSwap.java:

     100     99     101     101     100     100     99     98     101     102     104     106     96     99     100     103     102     101     96     103

18.10.2 MUTEX for Dealing with POSIX Thread Interference

Thread interference in POSIX threads can be eliminated by using a mutex lock. The name mutex, an abbreviation of mutual exclusion, is one of the two thread synchronization primitives provided by POSIX, the other being the condition variable presented in the next subsection. The idea behind a mutex lock is that a shared resource—such as a global variable—after it is locked by invoking pthread_mutex_lock is accessible to only that thread which invoked the lock. The resource does not become available to any other thread until the locking thread releases thelock by invoking pthread_mutex_unlock.

In the following program, line (A) declares the identifier mutex as a mutex lock. To initialize a mutex, we can either use the system supplied defaults, as we do in the commented out line (E), or we can declare a mutex attribute object for the purpose of initialization, as we do in line (B). The mutex attribute object is initialized to its defaults by the invocation shown in line (C). In Linux, the mutex attribute object consists of only one attribute—the mutex kind, which can either be PTHREAD_MUTEX_FAST_NP orPTHREAD_MUTEX_RECURSIVE_NP, the former being the default value of the attribute.^[21] The mutex kind determines what happens if a thread invokes pthread_mutex_lock on a mutex that it had locked previously. If a mutex is of the "recursive" kind, the invocation of the locking function succeeds. However, if a mutex is of the "fast" kind, the calling thread is suspended forever. For the recursive kind, the mutex records the number of times the thread owning the mutex has locked it through recursive calls. The owning thread must call the pthread_mutex_unlock function equal number of times before the mutex returns to the unlocked state. The kind of a mutex can be set by invoking

     int pthread_mutexattr_setkind_np(pthread_mutexa    ttr_t* attr,int kind)

and the kind of a previously created mutex can be examined by invoking

     int pthread_mutexattr_getkind_np(pthread_mutexa    ttr_t* attr,int* kind)

Both of these functions return 0 when they execute successfully. For the'get’ function, if the kind of a mutex is what is specified by the second argument, it returns successfully with 0.

Getting back to the problem of fixing thread interference in our earlier example of UnsynchedSwaps.c, we now place the vulnerable-to-interference parts of the code in the itemSwap and the test functions between the invocation pthread_mutex_lock(&mutex) and the invocation pthread_mutex_unlock(&mutex), as shown in lines (F), (G), (H), and (I). This way only one thread at a time will be able to execute these functions.

 
 //SynchedSwaps.c #include <pthread.h> #include <stdio.h> #include <time.h> pthread_mutex_t mutex;                                                 //(A) pthread_mutexattr_t attr;                                              //(B) typedef struct {     int dataItem1;     int dataItem2; } DataObject; DataObject dobj = { 50, 50 }; void keepBusy(double howLongInMillisec); void itemSwap(DataObject* dptr); void test(DataObject* dptr); void repeatedSwaps(DataObject* dptr); main() {     pthread_t t1, t2, t3, t4;     pthread_mutexattr_init(&attr);                                    //(C)     pthread_mutex_init(&mutex, &attr);                                //(D)     // pthread_mutex_init(&mutex, NULL);                              //(E)        pthread_create(&t1, NULL, (void* (*)(void*)) repeatedSwaps, &dobj);   pthread_create(&t2, NULL, (void* (*)(void*)) repeatedSwaps, &dobj);   pthread_create(&t3, NULL, (void* (*)(void*)) repeatedSwaps, &dobj);   pthread_create(&t4, NULL, (void* (*)(void*)) repeatedSwaps, &dobj);   pthread_join(t1, NULL);   pthread_join(t2, NULL);   pthread_join(t3, NULL);   pthread_join(t4, NULL); } void keepBusy(double howLongInMillisec) {     int ticksPerSec = CLOCKS_PER_SEC;     int ticksPerMillisec = ticksPerSec / 1000;     clock_t ct = clock();     while (clock() < ct + howLongInMillisec * ticksPerMillisec)         ; } void itemSwap(DataObject* dptr) {     int x;     pthread_mutex_lock(&mutex);                                        //(F)     x = (int) (-4.999999 + rand() % 10);     dptr->dataItem1 -= x;     keepBusy(10);     dptr->dataItem2 += x;     pthread_mutex_unlock(&mutex);                                      //(G) } void test(DataObject* dptr) {     int sum;     pthread_mutex_lock(&mutex);                                        //(H)     sum = dptr->dataItem1 + dptr->dataItem2;     printf("%d\n", sum);     pthread_mutex_unlock(&mutex);                                      //(I) } void repeatedSwaps( DataObject* dptr) {     int i = 0;     while (i < 20000) {         itemSwap(dptr);         if (i % 4000 == 0) test(dptr);         keepBusy(1); // in milliseconds         i++;     } }

When a mutex lock is no longer needed, it can be destroyed by invoking pthread_mutex_destroy() with an argument that is a pointer to the mutex lock.

18.10.3 POSIX Threads: Condition Variables and the wait-signal Mechanism for Dealing with Deadlock

Using POSIX threads, potential deadlock can often be avoided by thread synchronization using condition variables. A condition variable acts like a signaling object. When a thread cannot continue execution because a certain data condition is not satisfied, it calls a special function that suspends the execution of the thread and at the same time sets up a signaling object known as a condition variable. And then later when some other thread is able to create the data conditions necessary for the first thread to proceed, the second thread signals the condition variable, enabling the first thread to try to regain its lock on the processor.

A condition variable is initialized by

     pthread_cond_t cv;     cv = malloc(sizeof(pthread_cond_t));     pthread_cond_init(cv, NULL);                                      //(A)

In general, the initialization is carried out by invoking

     int pthread_cond_init(pthread_cond_t* cond,     pthread_condattr_t* attr);                                        //(B)

which initializes the condition variable cond using the attributes specified in attr. Default attributes are used if attr is NULL, as in line (A) above. The LinuxThreads implementation does not support any attributes for condition variables and ignores the second argument in the call to the initializer in line (B).

Apart from the initialization, the most commonly used condition variable related functions are

     int pthread_cond_wait(pthread_cont_t* cond, pthread_mutex_t* mutex);

and

     int pthread_cond_signal(pthread_cond_t* cond);

The first function, pthread_cond_wait, causes the unlocking of the mutex that is supplied to it as the second argument. This unlocking acts in the same manner as if a call was made to pthread_mutex_unlock. Additionally, and even more importantly, the thread execution is suspended until the condition variable cond is signaled, perhaps by the action of some other thread. It must be the case that the mutex that is unlocked by pthread_cond_wait was locked previously by the the same thread that calls pthread_cond_wait—the calling thread. When the condition variable is signaled, pthread_cond_wait re-acquires the lock (as per pthread_mutex_lock).

The thread state created by the execution of pthread_cond_wait can be thought of as the calling thread waiting on the condition variable that is the first argument to the function. The wait is for the condition variable to be signaled. In the meantime, the calling thread stays suspended.

A condition variable on which a thread is waiting is signaled by the execution of pthread_cond_signal by some other thread. It is possible for multiple threads to be waiting on the same condition variable. When such a condition variable is signaled, only one of those threads will be restarted. The POSIX standard does not say which one. If no threads are waiting on a condition variable that is signaled by some other thread, nothing happens.

If it is desired to simultaneously restart all the threads waiting on the same condition variable, the function to use is pthread_cond_broadcast:

     int pthread_cond_broadcast(pthread_cond_t* cond);

Other condition variable functions that we do not present here are pthread_cond_timedwait and pthread_cond_destroy.

The program shown below is a POSIX threads version of the MultiCustomerAccount Java class of Section 18.6.^[22] To make sure that there are sufficient funds in the common account before a withdrawal can take place, the program uses the wait-signal mechanism of POSIX threads. If the amount to be withdrawn is greater than the current balance, the following statement is executed in line (C) of the withdraw function:

     pthread_cond_wait(&cv, &mutex);

where cv is the condition variable that is used in conjunction with the mutex lock. This statement causes the thread to relinquish its lock on the code block and places the thread back in the wait state, where it waits for the condition variable to be signaled by the execution of

     pthread_cond_broadcast(&cv);

by a depositor thread in line (A). When the signal is received, the waiting thread is put back in the running for time on the processor. When it starts executing again, the test in the while loop of line (B) causes it to recheck that the amount to be withdrawn does not exceed the new balance in the account. As long as this condition is not satisfied, it keeps on executing the wait function and relinquishing its mutex lock. But when the condition is satisfied, it goes ahead with the withdrawal. In this manner, the balance is never allowed to become negative.

 
 //MultiCustomerAccount.c #include <pthread.h> #include <stdio.h> #include <stdlib.h> #include <time.h> void keepBusy(double howLongInMillisec); pthread_mutex_t mutex; pthread_mutexattr_t attr; pthread_cond_t cv; typedef struct {     int balance; } Account; Account* create_account(); void deposit(Account* a, int dep); void withdraw(Account* a, int draw); void multiple_deposits(Account* acct); void multiple_withdrawals(Account* acct); main() {     int i;     int status;     pthread_t depositorThreads[5];     pthread_t withdrawerThreads[5];     Account* account = create_account();     pthread_mutexattr_init(&attr);     pthread_mutex_init(&mutex, &attr);     pthread_cond_init(&cv, NULL);     for (i=0; i < 5; i++) {                 pthread_create( depositorThreads + i,                                 NULL                                 (void*(*) (void*)) multiple_deposits,                               account);                 pthread_create(withdrawerThreads + i,                                 NULL,                                 (void*(*) (void*)) multiple_withdrawals,                            account);     }     for (i=0; i < 5; i++) {         pthread_join(*(depositorThreads + i), (void*) &status);         pthread_join(*(withdrawerThreads + i), (void*) &status);     } } Account* create_account() {     Account* a = malloc(sizeof(Account));     a->balance = 0;     return a; } void deposit(Account* a, int dep) {     pthread_mutex_lock(&mutex);     a->balance += dep;     pthread_cond_broadcast(&cv);                                      //(A)     pthread_mutex_unlock(&mutex); } void withdraw(Account* a, int draw) {     pthread_mutex_lock(&mutex);     while (a->balance < draw) {                                       //(B)         pthread_cond_wait(&cv, &mutex);                               //(C)     }     a->balance -= draw;     pthread_mutex_unlock(&mutex); } void multiple_deposits(Account* acct) {     int i = 0;     int x;     while (1) {         x = rand() % 10;         deposit(acct, x);         if (i++ % 100 == 0)             printf("balance after deposits: %d\n", acct->balance);         keepBusy(1);     } } void multiple_withdrawals(Account* acct) {     int x;     int i = 0;     while (1) {         x = rand() % 10;         withdraw(acct, x);         if (i++ % 100 == 0)             printf("balance after withdrawals: %d\n", acct->balance);         keepBusy(1);     } } void keepBusy(double howLongInMillisec) {     int ticksPerSec = CLOCKS_PER_SEC;     int ticksPerMillisec = ticksPerSec / 1000;     clock_t ct = clock();     while (clock() < ct + howLongInMillisec * ticksPerMillisec)         ; }

Here is a portion of the output produced by the program. As the reader can see, the use of the wait-signal mechanism on the condition variable prevents the account balance from going negative. If a withdrawer thread wants to withdraw an amount that exceeds the balance in the account, it waits until the one or more depositor threads have created a sufficiently large balance in the account.

     balance after deposits: 3     balance after withdrawals: 2     balance after deposits: 10     balance after withdrawals: 21     balance after deposits: 63     balance after withdrawals: 63     balance after deposits: 66     balance after withdrawals: 81     balance after deposits: 86     balance after withdrawals: 285     balance after deposits: 331     balance after withdrawals: 325     balance after withdrawals: 221     balance after deposits: 232     balance after withdrawals: 281     ....     ....

^[15]POSIX, which stands for Portable Operating System Interface, can be thought of as a portable subset of Unix-like operating systems. It is an IEEE standard that was designed to facilitate the writing of programs that would run on the operating systems from different vendors. The specific standard that deals with thread management is POSIX.4.

^[16]The call to pthread_create shown here is meant to illustrate the basic syntax of the function call. This function, as all other "pthread" functions, returns an error code, which is 0 for successful completion and some nonzero value in case of failure. For functions returning error codes, it is a good programming practice to trap the returned value and to then print out any errormessages by using a library function like strerror (errorcode) that returns a string corresponding to its argument.

^[17]As an alternative to NULL for the second argument, the default-value initialization of all thread attributes can also be specified by using an object of type pthread_attr_t on which the function pthread_attr_init has been invoked, as we do in the program ThreadBasic.c of this section.

^[18]These are reproduced from the man pages for LinuxThreads by Xavier Leroy, who is also the original developer of LinuxThreads.

^[19]Like Java's join whose use was illustrated in Section 18.1, pthread_join suspends the calling thread until the called thread has run to completion. In addition to not being callable on a detached thread, pthread_join can also not be called on a daemon thread. A daemon thread, like the garbage collection thread in Java, runs in the background and the termination of a program is not predicated upon the daemon thread running to completion. We will show anexample later in this section that uses pthread_join.

^[20]The above program assumes that the sleep command puts an individual thread to sleep. That is the case with Linux, which was the environment on the machine on which the above program was tested. Also, note the importance of flushing the output buffer in line (C) in the function executed by the threads. If we deliberately introduce additional time delays between the different threads, you'd not be able to see the effect of thoseadditional delays unless the output buffer is flushed in the manner shown.

^[21]The NP suffix means that it is a nonportable extension to the POSIX standard and should not be used in code meant to be portable.

^[22]The overall logic of the program MultiCustomerAccount.c of this section is the same asthat of MultiCustomerAccount.java of Section 18.6. See the Java description for a more detailed explanation of the logic.