Hack 52 Distributed Command Execution

Use tentakel for parallel, distributed command execution.

Often you want to execute a command not only on one computer, but on several at once. For example, you might want to report the current statistics on a group of managed servers or update all of your web servers at once.

5.12.1 The Obvious Approach

You could simply do this on the command line with a shell script like the following:

# for host in hostA hostB hostC > do ssh $host do_something > done

However, this has several disadvantages:

• It is slow because the connections to the remote hosts do not run in parallel. Every connection must wait for the previous one to finish.

• Managing many sets of hosts can become a complicated task because there is no easy way to define groups of hosts (e.g., mailservers or workstations).

• The output is provided by the program that is run remotely.

• The output is hard to read because there are no marks indicating when the output for a specific host begins or ends.

5.12.2 How tentakel Can Help

While you could write a shell script to address some of these disadvantages, you might want to consider tentakel, which is available in the ports collection. Its execution starts multiple threads that run independently of each other. The maximum waiting time depends on the longest running remote connection, not on the sum of all of them. After the last remote command has returned, tentakel displays the results of all remote command executions. You can also configure how the output should look, combining or differentiating the results from individual hosts.

tentakel operates on groups of hosts. A group can have two types of members: hosts or references to other groups. A group can also have parameters to control various aspects of the connection, including username and access method (rsh or ssh, for example).

5.12.3 Installing and Configuring tentakel

Install tentakel from the ports collection:

# cd /usr/ports/sysutils/tentakel # make install clean

You can instead install tentakel by hand; consult the INSTALL file in the distribution. A make install should work in most cases, provided that you have a working Python environment installed.

After the installation, create the configuration file tentakel.conf in the directory $HOME/.tentakel/. See the example file in /usr/local/share/doc/tentakel/tentakel.conf.example for a quick overview of the format.

Alternatively, copy the file into /usr/local/etc/ or /etc/, depending on your system's policy, in order to have a site-wide tentakel.conf that will be used when there is no user-specific configuration. As an administrator, you may predefine groups for your users this way.

Assuming that you have a farm of three servers, mosel, aare, and spree, of which the first two are web servers, your configuration might resemble this:

set format="%d\n%o\n" group webservers(user="webmaster")   +mosel +aare group servers(user="root")   @webservers +spree

With this definition, you can use the group name servers to execute a command on all your servers as root and the group name webservers to execute it only on your web servers as user webmaster.

The first line defines the output format, as explained in Figure 5-1.

Table 5-5. tentakel output format characters

Character	Output
%d	The hostname
%o	The output of the remotely executed commands
\n	A newline character

This commands tentakel to print the hostname, followed by the lines of the remote output for each server sequentially. You can enrich the format string with additional directives, such as %s for the exit status from commands. See the manpage for more information.

As you can see from the servers definition, there is no need to list all servers in each group; include servers from other groups using the @groupname notation.

On the remote machines, the only required configuration is to ensure that you can log into them from the tentakel machine without entering a password. Usually that will mean using ssh and public keys, which is also tentakel's default. tentakel provides the parameter method for using different mechanisms, so refer to the manpage for details.

5.12.4 Using tentakel

To update the web pages on all web servers from a CVS repository:

% tentakel -g webservers "cd /var/www/htdocs && cvs update" ### mosel(0): cvs update: Updating . U index.html U main.css ### aare(1): C main.css cvs update: Updating . %

Note the use of quotes around the command to be executed. This prevents the local shell from interpreting special characters such as & or ;.

If no command is specified, tentakel invokes interactive mode:

% tentakel  interactive mode tentakel(default)> use webservers tentakel(webservers)> exec du -sh /var/www/htdocs ### mosel(0): 364k    /var/www/htdocs ### aare(0): 364k    /var/www/htdocs tentakel(webservers)> quit %

While in interactive mode, the command help prints further information.

5.12.5 See Also

• man tentakel

• The tentakel web site (http://tentakel.biskalar.de/)