Encoding and Decoding Arbitrary Data Types

For RPCs to pass data between systems with differing architectures, data is first converted to a standard XDR format. The conversion from a native representation to XDR format is called serialization . When the data is received in XDR format, it is converted back to the native representation for the recipient process. The conversion from XDR format to native format is called deserialization . To be transparent, the conversion process must take into account such things as native byte order, [11] integer size , representation of floating-point values, and representation of character strings. Some of these differences may be hardware-dependent, while others may be programming language-dependent. Once the data is converted, it is assumed that individual bytes of data (each consisting of eight bits) are in themselves portable from one platform to another. Data conversion for standard simple data types (such as integers, floats, characters , etc.) are implemented via a series of predefined XDR primitive type library routines, [12] which are sometimes called filters . These filters return the simple data type if they successfully convert the data; otherwise , they return a value of 0. Each primitive routine takes a pointer to the result and a pointer to the XDR handle. The primitive routines are incorporated into more complex routines for complex data types, such as arrays. The specifics of how the data conversion is actually done, while interesting, is beyond the scope of a one-chapter overview of RPC. However, it is important to keep in mind that such routines are necessary and that when passing data with RPCs, the proper conversion routines must be selected. Fortunately, when using rpcgen , the references for the appropriate XDR conversion routines are automatically generated and placed in another C source file the application can reference. This file, containing the conversion routines for both the client and server, will have the suffix _xdr.c appended to the protocol definition file name .

[11] For example, with a 4-byte (32-bit) number, the most significant byte (MSB) is always leftmost and the least significant byte (LSB) rightmost. If the sequence of bytes composing the number is ordered from left to right, as in the SPARC, the order is called big endian. If the byte sequence is numbered from right to left, as in the i80x86 processor line, the order is called little endian.

[12] For more details, see the manual pages on xdr .

To illustrate how data conversion is done, we will create an application that performs a remote directory tree listing. When the server for this application is passed a valid directory name, it will traverse the indicated directory and produce an indented listing of all of the directory's underlying subdirectories. For example, say we have the directory structure shown in Figure 9.28.

Figure 9.28. A hypothetical directory structure.

If we request the application to produce a directory tree of the directory /usr0 , the output returned from the directory tree application would be similar to that shown in Figure 9.29.

Figure 9.29 The directory tree listing of /usr0 .

/usr0:
 home
 joe
 bill
 prgm
 ex0
 ex1
 ex2

The directory traversed is listed with a trailing colon . Following this, each subdirectory displayed is indented to indicate its relationship to its parent directory and sibling subdirectories. The subdirectories home and prgm are at the same level and thus are indented the same number of spaces. The subdirectories joe and bill , which are beneath the home directory, are indented to the same level as are the subdirectories ex0 , ex1 , and ex2 , which are beneath the prgm directory.

As written, the application will pass from the client to the server the name of the directory to be traversed on the server. The server will allocate an array of a fixed size [13] to store the tree directory listing and will return the contents of the array if it is successful. If the server fails, it will return a NULL value. The server using the following high-level algorithm fills the array with the directory tree information.

[13] I know, I know, this is not the best way to do thisa dynamic allocation would be more appropriate here, as we do not know in advance how much storage room we will actually need. What is presented is a pedagogical example. The modification of the text example to use dynamic memory allocation is addressed in the exercise section.

The passed directory reference is opened. While the directory reference is not NULL, each entry is checked to determine if it is accessible. Note that if the server process does not have root privileges, this may cause some entries to be skipped . If the entry is accessible and is a directory reference (versus a file reference) but not a dot entry (we are looking to skip the "." and ".." entries for obvious reasons), the entry is stored with the proper amount of indenting in the allocated array. For display purposes, each stored entry has an appended newline ( n ) to separate it from the following entry. Since directory structures are recursive in nature, after processing an accessible directory entry, the tree display routine will call itself again, passing the name of the new directory entry. Once the entire contents of a directory have been processed, the directory is closed. When all directories and subdirectories have been processed , the array, with the contents of the directory tree, is returned to the client process, which will display its contents. The partial contents of the array returned for the previous example is shown in Figure 9.30.

Figure 9.30. A partial listing of the directory tree array for /usr0 .

The protocol definition file tree.x for the tree program is shown in Figure 9.31.

Figure 9.31 The protocol definition file, tree.x .

File : tree.x
 /*
 Tree protocol definition file
 */
 const MAXP = 4096; /* Upper limit on packet size it 8K. */
 + const DELIM = "
"; /* To separate each directory entry. */
 const INDENT= 5; /* # of spaces to indent for one level. */
 const DIR_1 = 128; /* Maximum length of any one directory
 entry. */
 
 typedef char line[MAXP]; /* A large storage location for all the
 entries */
 10 typedef line *line_ptr; /* A reference to the storage location. */

 /* If no errors return reference else
 return void */
 union dir_result
 switch( int errno ) {
 case 0:
 + line *line_ptr;
 default:
 void;
 };
 /*
 20 * The do_dir procedure will take a reference to a directory and return
 * a reference to an array containing the directory tree.
 */
 program TREE {
 version one{
 + dir_result do_dir( string ) = 1;
 } = 1;
 } = 0x2000001;

In the protocol definition file there is a series of constants. These constants will be mapped into #define statements by the rpcgen compiler. Following the constant section are two type declarations. The first, typedef char line[MAXP] , declares a new type called line that is an array of MAXP number of characters. To translate this type, rpcgen creates a routine named xdr_line , which it places in the tree_xdr.c file. The contents of this routine are shown in Figure 9.32.

Figure 9.32 The xdr_line XDR conversion routine created by rpcgen .

File : tree_xdr.c
 /*
 * Please do not edit this file.
 * It was generated using rpcgen.
 */
 +
 #include "tree.h"
 
 bool_t
 xdr_line (XDR *xdrs, line objp)
 10 {
 register int32_t *buf;
 
 if (!xdr_vector (xdrs, (char *)objp, MAXP,
 sizeof (char), (xdrproc_t) xdr_char))
 + return FALSE;
 return TRUE;
 }
 . . .

The generated xdr_line routine calls the predefined xdr_vector routine, which in turn invokes the xdr_char primitive. It is the xdr_char primitive that is found in both the client and server stub files that does the actual translation. A similar set of code is generated for the line pointer ( line_ptr ) declaration and the discriminated union that declares the type to be returned by the user -defined remote do_dir procedure. If we examine the tree.h file produced by rpcgen from the tree.x file, we find the discriminated union is mapped to a structure that contains a union (as shown in Figure 9.33). The single argument for the do_dir procedure is a string (a special XDR data type), which is mapped to a pointer to a pointer to a character. The argument to do_dir will be the directory to examine .

Figure 9.33 Structure, found in tree.h , generated by rpcgen from the discriminated union in tree.x .

File : tree.h
 /*
 * Please do not edit this file.
 * It was generated using rpcgen.
 */
 +
 #ifndef _TREE_H_RPCGEN
 #define _TREE_H_RPCGEN
 
 #include 
 ...
 typedef line *line_ptr;

+ struct dir_result {


int errno;


union {


line *line_ptr;


} dir_result_u;


30 };

typedef struct dir_result dir_result;
 ...

The code for the client portion ( tree_client.c ) of the tree program is shown in Program 9.6, and the server portion ( tree_server.c ) is shown in Program 9.7.

Program 9.6 The directory tree client program tree_client.c.

File : tree_client.c
 /*
 
 ##### ##### ###### ###### #### # # ###### # # #####
 # # # # # # # # # # ## # #
 + # # # ##### ##### # # # ##### # # # #
 # ##### # # # # # # # # # #
 # # # # # # # # # # # ## #
 # # # ###### ###### #### ###### # ###### # # #
 */
 10 #include "local.h"
 #include "tree.h"
 
 void
 tree_1(char *host, char *the_dir ) {
 + CLIENT *client;
 dir_result *result;
 
 #ifndef DEBUG
 client = clnt_create(host, TREE, one, "tcp");
 20 if (client == (CLIENT *) NULL) {
 clnt_pcreateerror(host);
 exit(2);
 }
 result = do_dir_1(&the_dir, client);
 + #else
 result = do_dir_1_svc(&the_dir, (svc_req *) client);
 #endif /* DEBUG */
 if (result == (dir_result *) NULL) {
 #ifndef DEBUG
 30 clnt_perror(client, "call failed");
 #else
 perror("Call failed");
 #endif /* DEBUG */
 exit(3);
 + } else /* display the whole array */
 printf("%s:

%s
",the_dir,result->dir_result_u.line_ptr);
 #ifndef DEBUG
 clnt_destroy(client);
 #endif /* DEBUG */
 40 }
 int
 main(int argc, char *argv[]) {
 char *host;
 static char directory[DIR_1]; /* Name of the directory */
 + if (argc < 2) {
 fprintf(stderr, "Usage %s server [directory]
", argv[0]);
 exit(1);
 }
 host = argv[1]; /* Assign the server */
 50 if (argc > 2)
 strcpy(directory, argv[2]);
 else
 strcpy(directory , ".");
 tree_1(host, directory); /* Give it a shot! */
 + return 0;
 }

The bulk of the tree_client.c program contains code that is either similar in nature to previous RPC examples or is self-documenting . The one statement that may bear further explanation is the printf statement that displays the directory tree information to the screen. Remember that the remote procedure returns a pointer to a string . This string is already in display format in that each directory entry is separate from the next with a newline. The reference to the string is written as result->dir_result_u.line_ptr . The proper syntax for this reference is obtained by examining the tree.h file produced by rpcgen .

Program 9.7 The directory tree client program tree_server.c .

File : tree_server.c
 /*
 ##### ##### ###### ###### #### ###### ##### # # ###### #####
 # # # # # # # # # # # # # #
 # # # ##### ##### #### ##### # # # # ##### # #
 + # ##### # # # # ##### # # # #####
 # # # # # # # # # # # # # # #
 # # # ###### ###### #### ###### # # ## ###### # #
 */
 #include "local.h"
 10 #include "tree.h"
 
 static int cur = 0, /* Index into output array */
 been_allocated = 0, /* Has array been allocated? */
 depth = 0; /* Indenting level */
 +
 dir_result *
 do_dir_1_svc( char **f, struct svc_req * rqstp) {
 static dir_result result; /* Either array or void */
 struct stat statbuff; /* For status check of entry */
 20 DIR *dp; /* Directory entry */
 struct dirent *dentry; /* Pointer to current entry */
 char *current; /* Position in output array */
 int length; /* Length of current entry */
 static char buffer[DIR_1]; /* Temp storage location */
 +
 if (!been_allocated) /* If not done then allocate */
 if ((result.dir_result_u.line_ptr=(line *)malloc(sizeof(line))) == NULL)
 return (&result);
 else{
 30 been_allocated = 1; /* Record allocation */
 } else if ( depth == 0 ) { /* Clear 'old' contents. */
 memset(result.dir_result_u.line_ptr, 0, sizeof(line));
 cur = 0; /* Reset indent level */
 }
 + if ((dp = opendir(*f)) != NULL) { /* If successfully opened */
 chdir(*f); /* Change to the directory */
 dentry = readdir(dp); /* Read first entry */
 while (dentry != NULL) {
 if (stat(dentry->d_name, &statbuff) != -1) /* If accessible */
 40 if ((statbuff.st_mode & S_IFMT) == S_IFDIR && /* & a directory */
 dentry->d_name[0] != '.') { /* & not . or .. */
 depth += INDENT; /* adjust indent */
 /*
 Store the entry in buffer - then copy buffer into larger array.
 + */
 sprintf(buffer, "%*s %-10s
", depth, " ", dentry->d_name);
 length = strlen(buffer);
 memcpy((char *)result.dir_result_u.line_ptr + cur, buffer, length);
 cur += length; /* update ptr to ref next loc */
50 current = dentry->d_name; /* the new directory */
 (dir_result *)do_dir_1_svc(¤t, rqstp); /* call self */
 chdir(".."); /* back to previous level */
 depth -= INDENT; /* adjust the indent level */
 }
 + dentry = readdir(dp); /* Read the next entry */
 }
 closedir(dp); /* Done with this one */
 }
 return (&result); /* Pass back the result */
60 }

In the tree_server.c program, there are several static integers that are used either as counters or flags. The cur identifier references the current offset into the output array where the next directory entry should be stored. Initially, the offset is set to 0. The been_allocated identifier acts as a flag to indicate whether or not an output buffer has been allocated. Initially, this flag is set to 0 (FALSE). The last static identifier, depth , is used to track the current indent level. It is also set to 0 at the start.

The do_dir_1_svc procedure is passed a reference to a string (actually a character array) and a reference to an RPC client handle. Within the procedure, a series of local identifiers are allocated to manipulate and access directory information. Following this is an if statement that is used to test the been_allocated flag. If an output buffer has not been allocated, a call to malloc generates it. The allocated buffer is cast appropriately and is referenced by the line_ptr member of the dir_result_u structure. Once the buffer has been allocated, the been_allocated flag is set to 1 (TRUE). If the output buffer has already been allocated and this is the first call to this procedure (i.e., depth is at 0; remember this is a recursive procedure), a call to memset is used to clear the previous output contents by filling the buffer with NULL values. When the contents of the output buffer are cleared, the cur index counter is reset to 0.

The procedure then attempts to open the referenced directory. If it is successful, a call to chdir is issued to change the directory (this was done to eliminate the need to construct and maintain a fully qualified path when checking the current directory). Next, the first entry for the directory is obtained with the readdir function. A while loop is used to cycle through the directory entries. Those entries for which the process has access permission are tested to determine if they reference a directory. If they do, and the directory does not start with a dot ( . ), the depth counter is incremented. The formatted directory entry is temporarily constructed in a buffer using the sprintf string function. The format descriptors direct sprintf to use the depth value as a dynamic indicator of the number of blanks it should insert prior to the directory entry. Each entry has a newline appended to it. The formatted entry is then copied (using memcpy ) to the proper location in the output buffer using the value of cur as an offset. The directory name is then passed back to the do_dir_1_svc procedure via a call to itself. Upon return from parsing a subdirectory, the procedure returns up one level via a call to chdir and decrements the depth counter accordingly . Once the entire directory is processed, the directory file is closed. When the procedure finishes, it returns the reference to output buffer.

An output sequence for the directory tree clientserver application is shown in Figure 9.34. In this example, the directory tree server, tree_server , is run on the host called kahuna . The user, on host medusa , runs the tree_client program, passing the host name kahuna and the directory /usr/bin . The output, shown on the host medusa , is the directory tree found on kahuna (where the tree_server program is running in the background).

Figure 9.34 A sample run of the directory tree application.

medusa$ tree_client kahuna /usr/bin
/usr/bin:
 X11
 man
 man1
 man4
 man5
 man7
 man6
 man3

EXERCISE

On most UNIX-based systems the spell utility uses a file of words as a base for its spell checking. On our Linux system this file is in the /usr/share/dict directory. Write an RPC-based clientserver application in which the client sends a word or partial word to the server. To process the request, the server returns all the words that contain the requested word (partial word) or the message "Nothing appropriate" if no match can be made.

EXERCISE

Modify the directory tree example so that the server process allocates , on the fly, a node (structure) for each directory entry. A list of the nodes should be returned to the client (versus the fixed array in the example). Be sure to dispose of all allocated memory once the application is finished with it.