Section 17.6. An Open Server, Version 2

17.6. An Open Server, Version 2

In the previous section, we developed an open server that was invoked by a fork and exec by the client, demonstrating how we can pass file descriptors from a child to a parent. In this section, we develop an open server as a daemon process. One server handles all clients. We expect this design to be more efficient, since a fork and exec are avoided. We still use an s-pipe between the client and the server and demonstrate passing file descriptors between unrelated processes. We'll use the three functions serv_listen, serv_accept, and cli_conn introduced in Section 17.2.2. This server also demonstrates how a single server can handle multiple clients, using both the select and poll functions from Section 14.5.

The client is similar to the client from Section 17.5. Indeed, the file main.c is identical (Figure 17.27). We add the following line to the open.h header (Figure 17.26):

 #define CS_OPEN "/home/sar/opend" /* server's well-known name */ 

The file open.c does change from Figure 17.28, since we now call cli_conn instead of doing the fork and exec. This is shown in Figure 17.34.

Figure 17.34. The csopen function, version 2

 #include    "open.h" #include    <sys/uio.h>     /* struct iovec */ /*  * Open the file by sending the "name" and "oflag" to the  * connection server and reading a file descriptor back.  */ int csopen(char *name, int oflag) {     int             len;     char            buf[10];     struct iovec    iov[3];     static int      csfd = -1;     if (csfd < 0) {     /* open connection to conn server */         if ((csfd = cli_conn(CS_OPEN)) < 0)             err_sys("cli_conn error");     }     sprintf(buf, " %d", oflag);     /* oflag to ascii */     iov[0].iov_base = CL_OPEN " ";  /* string concatenation */     iov[0].iov_len  = strlen(CL_OPEN) + 1;     iov[1].iov_base = name;     iov[1].iov_len  = strlen(name);     iov[2].iov_base = buf;     iov[2].iov_len  = strlen(buf) + 1;  /* null always sent */     len = iov[0].iov_len + iov[1].iov_len + iov[2].iov_len;     if (writev(csfd, &iov[0], 3) != len)         err_sys("writev error");     /* read back descriptor; returned errors handled by write() */     return(recv_fd(csfd, write)); } 

The protocol from the client to the server remains the same.

Next, we'll look at the server. The header opend.h (Figure 17.35) includes the standard headers and declares the global variables and the function prototypes.

Figure 17.35. The opend.h header, version 2

 #include "apue.h" #include <errno.h> #define CS_OPEN "/home/sar/opend"   /* well-known name */ #define CL_OPEN "open"              /* client's request for server */ extern int   debug;     /* nonzero if interactive (not daemon) */ extern char  errmsg[];  /* error message string to return to client */ extern int   oflag;     /* open flag: O_xxx ... */ extern char *pathname;  /* of file to open for client */ typedef struct {    /* one Client struct per connected client */   int   fd;         /* fd, or -1 if available */   uid_t uid; } Client; extern Client   *client;        /* ptr to malloc'ed array */ extern int       client_size;   /* # entries in client[] array */ int      cli_args(int, char **); int      client_add(int, uid_t); void     client_del(int); void     loop(void); void     request(char *, int, int, uid_t); 

Since this server handles all clients, it must maintain the state of each client connection. This is done with the client array declared in the opend.h header. Figure 17.36 defines three functions that manipulate this array.

Figure 17.36. Functions to manipulate client array

 #include    "opend.h" #define NALLOC  10   /* # client structs to alloc/realloc for */ static void client_alloc(void)   /* alloc more entries in the client[] array */ {     int     i;     if (client == NULL)         client = malloc(NALLOC * sizeof(Client));     else         client = realloc(client, (client_size+NALLOC)*sizeof(Client));     if (client == NULL)         err_sys("can't alloc for client array");     /* initialize the new entries */     for (i = client_size; i < client_size + NALLOC; i++)         client[i].fd = -1;  /* fd of -1 means entry available */     client_size += NALLOC; } /*  * Called by loop() when connection request from a new client arrives.  */ int client_add(int fd, uid_t uid) {     int     i;     if (client == NULL)     /* first time we're called */         client_alloc(); again:     for (i = 0; i < client_size; i++) {         if (client[i].fd == -1) {   /* find an available entry */             client[i].fd = fd;             client[i].uid = uid;             return(i);  /* return index in client[] array */         }     }     /* client array full, time to realloc for more */     client_alloc();     goto again;     /* and search again (will work this time) */ } /*  * Called by loop() when we're done with a client.  */ void client_del(int fd) {     int     i;     for (i = 0; i < client_size; i++) {         if (client[i].fd == fd) {             client[i].fd = -1;             return;         }     }     log_quit("can't find client entry for fd %d", fd); } 

The first time client_add is called, it calls client_alloc, which calls malloc to allocate space for ten entries in the array. After these ten entries are all in use, a later call to client_add causes realloc to allocate additional space. By dynamically allocating space this way, we have not limited the size of the client array at compile time to some value that we guessed and put into a header. These functions call the log_ functions (Appendix B) if an error occurs, since we assume that the server is a daemon.

The main function (Figure 17.37) defines the global variables, processes the command-line options, and calls the function loop. If we invoke the server with the -d option, the server runs interactively instead of as a daemon. This is used when testing the server.

Figure 17.37. The server main function, version 2

 #include    "opend.h" #include    <syslog.h> int      debug, oflag, client_size, log_to_stderr; char     errmsg[MAXLINE]; char    *pathname; Client  *client = NULL; int main(int argc, char *argv[]) {     int     c;     log_open("open.serv", LOG_PID, LOG_USER);     opterr = 0;     /* don't want getopt() writing to stderr */     while ((c = getopt(argc, argv, "d")) != EOF) {         switch (c) {         case 'd':       /* debug */             debug = log_to_stderr = 1;             break;         case '?':             err_quit("unrecognized option: -%c", optopt);         }     }     if (debug == 0)         daemonize("opend");     loop();     /* never returns */ } 

The function loop is the server's infinite loop. We'll show two versions of this function. Figure 17.38 shows one version that uses select; Figure 17.39 shows another version that uses poll.

Figure 17.38. The loop function using select

 #include    "opend.h" #include    <sys/time.h> #include    <sys/select.h> void loop(void) {     int     i, n, maxfd, maxi, listenfd, clifd, nread;     char    buf[MAXLINE];     uid_t   uid;     fd_set  rset, allset;     FD_ZERO(&allset);     /* obtain fd to listen for client requests on */     if ((listenfd = serv_listen(CS_OPEN)) < 0)         log_sys("serv_listen error");     FD_SET(listenfd, &allset);     maxfd = listenfd;     maxi = -1;     for ( ; ; ) {         rset = allset;  /* rset gets modified each time around */         if ((n = select(maxfd + 1, &rset, NULL, NULL, NULL)) < 0)             log_sys("select error");         if (FD_ISSET(listenfd, &rset)) {             /* accept new client request */             if ((clifd = serv_accept(listenfd, &uid)) < 0)                 log_sys("serv_accept error: %d", clifd);             i = client_add(clifd, uid);             FD_SET(clifd, &allset);             if (clifd > maxfd)                 maxfd = clifd;  /* max fd for select() */             if (i > maxi)                 maxi = i;   /* max index in client[] array */             log_msg("new connection: uid %d, fd %d", uid, clifd);             continue;         }         for (i = 0; i <= maxi; i++) {   /* go through client[] array */             if ((clifd = client[i].fd) < 0)                 continue;             if (FD_ISSET(clifd, &rset)) {                 /* read argument buffer from client */                 if ((nread = read(clifd, buf, MAXLINE)) < 0) {                     log_sys("read error on fd %d", clifd);                 } else if (nread == 0) {                     log_msg("closed: uid %d, fd %d",                       client[i].uid, clifd);                     client_del(clifd);  /* client has closed cxn */                     FD_CLR(clifd, &allset);                     close(clifd);                 } else {    /* process client's request */                     request(buf, nread, clifd, client[i].uid);                 }             }         }     } } 

This function calls serv_listen to create the server's endpoint for the client connections. The remainder of the function is a loop that starts with a call to select. Two conditions can be true after select returns.

1. The descriptor listenfd can be ready for reading, which means that a new client has called cli_conn. To handle this, we call serv_accept and then update the client array and associated bookkeeping information for the new client. (We keep track of the highest descriptor number for the first argument to select. We also keep track of the highest index in use in the client array.)

2. An existing client's connection can be ready for reading. This means that the client has either terminated or sent a new request. We find out about a client termination by read returning 0 (end of file). If read returns a value greater than 0, there is a new request to process, which we handle by calling request.

We keep track of which descriptors are currently in use in the allset descriptor set. As new clients connect to the server, the appropriate bit is turned on in this descriptor set. The appropriate bit is turned off when the client terminates.

We always know when a client terminates, whether the termination is voluntary or not, since all the client's descriptors (including the connection to the server) are automatically closed by the kernel. This differs from the XSI IPC mechanisms.

The loop function that uses poll is shown in Figure 17.39.

Figure 17.39. The loop function using poll

 #include    "opend.h" #include    <poll.h> #if !defined(BSD) && !defined(MACOS) #include    <stropts.h> #endif void loop(void) {     int             i, maxi, listenfd, clifd, nread;     char            buf[MAXLINE];     uid_t           uid;     struct pollfd   *pollfd;     if ((pollfd = malloc(open_max() * sizeof(struct pollfd))) == NULL)         err_sys("malloc error");     /* obtain fd to listen for client requests on */     if ((listenfd = serv_listen(CS_OPEN)) < 0)         log_sys("serv_listen error");     client_add(listenfd, 0);    /* we use [0] for listenfd */     pollfd[0].fd = listenfd;     pollfd[0].events = POLLIN;     maxi = 0;     for ( ; ; ) {         if (poll(pollfd, maxi + 1, -1) < 0)             log_sys("poll error");           if (pollfd[0].revents & POLLIN) {             /* accept new client request */             if ((clifd = serv_accept(listenfd, &uid)) > 0)                 log_sys("serv_accept error: %d", clifd);             i = client_add(clifd, uid);             pollfd[i].fd = clifd;             pollfd[i].events = POLLIN;             if (i > maxi)                 maxi = i;             log_msg("new connection: uid %d, fd %d", uid, clifd);         }         for (i = 1; i <= maxi; i++) {             if ((clifd = client[i].fd) < 0)                 continue;             if (pollfd[i].revents & POLLHUP) {                 goto hungup;             } else if (pollfd[i].revents & POLLIN) {                 /* read argument buffer from client */                 if ((nread = read(clifd, buf, MAXLINE)) < 0) {                     log_sys("read error on fd %d", clifd);                 } else if (nread == 0) { hungup:                      log_msg("closed: uid %d, fd %d",                       client[i].uid, clifd);                     client_del(clifd);  /* client has closed conn */                     pollfd[i].fd = -1;                     close(clifd);                 } else {        /* process client's request */                     request(buf, nread, clifd, client[i].uid);                 }             }          }     } } 

To allow for as many clients as there are possible open descriptors, we dynamically allocate space for the array of pollfd structures. (Recall the open_max function from Figure 2.16.)

We use the first entry (index 0) of the client array for the listenfd descriptor. That way, a client's index in the client array is the same index that we use in the pollfd array. The arrival of a new client connection is indicated by a POLLIN on the listenfd descriptor. As before, we call serv_accept to accept the connection.

For an existing client, we have to handle two different events from poll: a client termination is indicated by POLLHUP, and a new request from an existing client is indicated by POLLIN. Recall from Exercise 15.7 that the hang-up message can arrive at the stream head while there is still data to be read from the stream. With a pipe, we want to read all the data before processing the hangup. But with this server, when we receive the hangup from the client, we can close the connection (the stream) to the client, effectively throwing away any data still on the stream. There is no reason to process any requests still on the stream, since we can't send any responses back.

As with the select version of this function, new requests from a client are handled by calling the request function (Figure 17.40). This function is similar to the earlier version (Figure 17.31). It calls the same function, buf_args (Figure 17.32), that calls cli_args (Figure 17.33), but since it runs from a daemon process, it logs error messages instead of printing them on the standard error stream.

Figure 17.40. The request function, version 2

 #include    "opend.h" #include    <fcntl.h> void request(char *buf, int nread, int clifd, uid_t uid) {     int     newfd;     if (buf[nread-1] != 0) {         sprintf(errmsg,           "request from uid %d not null terminated: %*.*s\n",           uid, nread, nread, buf);         send_err(clifd, -1, errmsg);         return;     }     log_msg("request: %s, from uid %d", buf, uid);     /* parse the arguments, set options */     if (buf_args(buf, cli_args) < 0) {         send_err(clifd, -1, errmsg);         log_msg(errmsg);         return;     }     if ((newfd = open(pathname, oflag)) < 0) {         sprintf(errmsg, "can't open %s: %s\n",           pathname, strerror(errno));         send_err(clifd, -1, errmsg);         log_msg(errmsg);         return;     }     /* send the descriptor */     if (send_fd(clifd, newfd) < 0)         log_sys("send_fd error");     log_msg("sent fd %d over fd %d for %s", newfd, clifd, pathname);     close(newfd);       /* we're done with descriptor */ } 

This completes the second version of the open server, using a single daemon to handle all the client requests.