Chapter 8: Toward a Message-Passing Library for Heterogeneous Networks of Computers

7.5. SUMMARY

Besides allowing the user to program computations and communications of the heterogeneous parallel algorithm, the mpC language allows the user to specify the performance model of the algorithm. This performance model takes into account the main features of the algorithm having an impact on its execution time, including

• the number of parallel processes executing the algorithm,

• the absolute volume of computations performed by each process,

• the absolute volume of data transferred between each pair of processes, and

• the scenario of interaction between the parallel processes during the execution of the algorithm.

This information is used at runtime to map the algorithm to the physical processors of the computer network. When perfoming the mapping, the mpC programming system tries to minimize the execution time of the algorithm.

The mpC language is designed with a common network of computers as the target parallel architecture. In such networks the performance of communication links and processors is rarely balanced for high-performance computing. Normally the ratio of the speed of data transfer to the speed of processors is significantly lower than that of specialized parallel computer systems. In addition the performance characteristics of common networks are usually not stable; indeed, performance can vary greatly because many independent users have access to their resources.

As a result it is very difficult—if at all possible—to make portable and efficient implementations of parallel algorithms using a fine and sophisticated structure of communications, or making communications prevail over computations, on common networks of computers. The mpC language is mainly suitable for programming parallel algorithms with a relatively simple and coarse-grained structure of communications, and allowing computations to prevail over communications.