6.3.2 Setting up a Clone Root Partition

		Now to run our simulation for every day of the year, we just run:

		prun /var/run/bnodes-up everyday

		It is no longer necessary to explicitly decide which processors will perform which simulations. The entire sequence is ''load balanced" by prun because whenever a processor finishes one frame, it is immediately given another one.

		7.3.3 Overheads-RSH and File I/O

		The strategy of running a program on multiple machines via rsh is not without costs. Generally, it takes longer to run a job remotely than to run it locally. This is due to the startup costs of establishing a connection to the remote host, authenticating the user, and exec'ing the remote command. Furthermore, there are additional overheads associated with I/O. If the executable file is stored on an NFS mounted file system, then the program itself may be transferred on every invocation. Similarly, any NFS-mounted data files that it requires or produces must be transferred over the network.

		The overhead associated with running a job remotely can be anywhere from a few tens of milliseconds to a few seconds. Anything more than that usually indicates that something is wrong with the system. NFS-based data transfers are usually limited by the performance of the server and its network connection. Even though the network switch might have an internal bandwidth in excess of 1 Gbps, the link to the NFS server is probably over a single 100 Mbps Ethernet connection. Thus, all simultaneous NFS accesses must contend for the shared 100 Mbps link to the server. In practice, these numbers should be compared with the typical time taken to run a remote job. Frequently the remote process performs a substantial computing task, taking anywhere from minutes to hours. In such cases, there is little point in worrying about a few extra seconds associated with startup and file transfer.

		For those cases in which startup and data transfer overheads are important, it is possible to mitigate the problem by pre-staging data and executables and storing results on the disks that are physically attached to the processor doing the computation. This is a situation in which the transparency of and convenience of the NFS file system can cause unwanted surprises. Although we recommend against it, your system administrator may configure a Byzantine maze of symbolic links, NFS mount points and automounter settings which makes it difficult to determine exactly where a file is located. A pathname like /usr/bin/foobar looks local, but may, in fact be a symbolic link to /packages/foo/bin/foobar, which may, in turn be NFS mounted on the worldly node. If in doubt, the -T flag to df will tell you how a file is mounted, e.g., df -T /shared/somefile.