| | | | | | 7.3.2 Utilities for Process-parallel Computing | | | | | | | | | A little experimentation with process-parallel computing using the techniques described above quickly reveals some major shortcomings. First, it is difficult to manually type all these commands accurately. Second, it is quite possible for different processes to take very different amounts of time, forcing one to wait for the slowest one, and significantly slowing down the overall execution. Third, we are exploiting POV-Ray's ability to loop over frames in a way that does not necessarily translate to other types of codes. For example, we might have a simulation that we wish to run many times with different input files. The existing simulation probably does not have controls analogous to POV-Ray's +SF and +EF flags. We may need a way to invoke this simulation hundreds or thousands of times, and it would be far too laborious to manually keep track of which processors are responsible for which invocations, etc. For these reasons, we have created a small "dispatcher" program that can take an arbitrary list of commands and dispatch them, one at a time, to a set of processors. | | | | | | | | | The Perl script in Programs 7.2 and 7.3 requires two arguments. The first is a filename containing a list of processor hostnames. These processors will be used as targets for remote execution commands. No more than one command will be run on any host at any one time, but the order in which commands are run, and the way they are dispatched to hosts is arbitrary. The second argument is a list of commands. One command is obtained from each non-blank line of the command file (following the conventions of shell programming, the '#' character is used to indicate comments that extend to the end of the line). Commands will be dispatched with the rsh utility, so absolute path names are still important. In the version of prun given, the standard input of the remote command is closed, and the standard output and standard error are redirected to files called prun-NNN. out and prun-NNN. err respectively, where NNN is the sequence number of the command in the script. In many cases, the remote commands themselves will be scripts which will perform suitable input and output redirection, in which case stdin and stdout could just as easily be closed. | | | | | | | | | For example, if we had a simulation that used daily weather data, and we created a script dosim to run it using the data for a given day, we might create a command file, everyday like: | | | | | | | | | ~/model/dosim 01jan97
~/model/dosim 02jan97
~/model/dosim 03jan97
...
~/model/dosim 31dec97
| | | | |
