Multiplexing I/O with select

Multiplexing I O with select

It is clear from the last example that when processes communicate, they need a way to coordinate their activities other than blocking (waiting) for the recipient process to respond. One approach is to change the socket from its default of blocking to non-blocking . The process could then perform its own polling/checking at some designated interval to determine if I/O is pending. This technique is shown in the modified server Program 10.12. The sections of code that have been added or significantly modified are in bold in the gray areas (lines 5, 6, 10, 1820, and 4249).

Program 10.12 Internet domain connectionless server , non-blocking.

File : p10.12.cxx
 /*
 Program 10.12 - SERVER Internet Domain - connectionless - NON-BLOCKING
 */
 #include "local_sock.h"

+ #include 


#include 

int
 main( ) {
 int sock, n,

10 errcount=0, flag=1;

socklen_t server_len, client_len;
 struct sockaddr_in server, // Internet Addresses
 client;
 // SOCKET
 + if ((sock = socket(PF_INET, SOCK_DGRAM, 0)) < 0) {
 perror("SERVER socket "); return 1;
 }

if (ioctl(sock, FIONBIO, &flag) < 0 ) {


perror("SERVER ioctl "); return 2;


20 }

memset(&server, 0, sizeof(server)); // Clear structure
 server.sin_family = AF_INET; // Set address type
 server.sin_addr.s_addr = htonl(INADDR_ANY);
 server.sin_port = htons(0);
 + // BIND
 if (bind(sock, (struct sockaddr *) &server,
 sizeof(server) ) < 0) {
 perror("SERVER bind "); return 3;
 }
 30 server_len = sizeof(server); // Obtain address length
 // Find picked port #
 if (getsockname(sock, (struct sockaddr *) &server,
 &server_len) < 0) {
 perror("SERVER getsocketname "); return 4;
 + }
 cout << "Server using port " << ntohs(server.sin_port) << endl;
 while ( 1 ) { // Loop forever
 client_len = sizeof(client); // estimate length
 memset(buf, 0, BUFSIZ); // clear the buffer
 40 if ((n=recvfrom(sock, buf, BUFSIZ, 0, // get the client's msg
 (struct sockaddr *) &client, &client_len)) < 0){

if ( errcount++ > 60 errno != EWOULDBLOCK ) {


perror("SERVER recvfrom ");


close(sock); return 5;


+ }


sleep(1);


continue;


}


errcount = 0;

50 write( fileno(stdout), buf, n ); // display msg on server
 memset(buf, 0, BUFSIZ); // clear the buffer
 if ( read(fileno(stdin), buf, BUFSIZ) != 0 ){// get server's msg
 if ((sendto(sock, buf, strlen(buf) ,0, // send to client
 (struct sockaddr *) &client, client_len)) <0){
 + perror("SERVER sendto ");
 close(sock); return 6;
 }
 }
 }
 60 return 0;
 }

In this example, the ioctl system call is used to change the socket to non-blocking. The ioctl call performs a wide variety of file control operations. [14] Its actions are described fully in two parts of the manual: ioctl and ioctl_list , both found in Section 2 of the manual pages. The ioctl call is not the only way to set the socket to non-blocking. An alternate approach is to use the fcntl (file control) system call. If fcntl is used, the syntax would be

[14] I realize that this is a departure from the normal approach (i.e., a full explanation of the system call once it is encountered /used). The ioctl system call is complex, and as we will be using it only in passing, the details of its syntax and use have been omitted.

#include  // include for fcntl call
#include 
. . .
if (fcntl(sock, F_SETFL, FNDELAY) < 0 ) { // new lines 18-20
 perror("SERVER fcntl "); return 2;
 }
. . .

In Program 10.12, the file is added to the include section. This file contains defined constants used by the ioctl system call. In the program we also reference errno and use one of the defined constants found in the include file . Once the socket is created, the ioctl call is employed to change the socket status to non-blocking. The ioctl call is passed the socket descriptor, the defined constant FIONBIO (signifying file I/O non-blocking I/O), and the address of an integer flag. If the ioctl call is successful, it returns a nonnegative value.

The processing loop of the server is modified to introduce a limited form of polling. If a message is not available, and fewer than 60 receive attempts have been made, the process will sleep and try again. When the socket is set to non-blocking, the recvfrom call returns immediately if no message is available. When this occurs, the external variable errno is set to EWOULDBLOCK, indicating the call would have blocked. As written, when the error code returned by recvfrom is EWOULDBLOCK, and the number of attempts to receive a message is less than 60, the process issues a call to sleep for one second. Once a message is received, the error count is reset to 0 and the message is processed as before. If recvfrom returns an error code other than EWOULDBLOCK, or the number of attempts to receive a message exceeds 60, an error message is generated and the process is exited. The number of times to retry and the amount of sleep time are arbitrary and can be adjusted by trial and error to meet specific needs.

While the above approach is both interesting and functional, it has all the drawbacks of any code that implements its own polling. It would seem that communication coordination would be greatly improved if a process could somehow notify a recipient process that a message was available. For example, we could signal a process when a socket has data to be read. To do this, the process receiving the signal must establish a signal handler for the SIGIO signal. Second, it must associate its process ID with the socket. Third, the socket must be set to allow asynchronous I/O. While all this is possible (using the signal and fcntl system calls), it too is less than desirable. Signals can get lost, and should multiple processes be involved in the communication process, coding can become quite complex. When possible, it is best to allow the system to handle the details of notifying processes that I/O is pending. The select library call can be used for this purpose. The select call, as shown in Table 10.25, is fairly complex.

Table 10.25. Summary of the select System Call.

Include File(s)	< sys/time.h>		Manual Section	2
Summary	int select( int n, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout );
Return	Success	Failure	Sets errno
	Number of ready file descriptors	-1	Yes

The select system call uses a series of file descriptor masks to determine which files it should check for pending I/O. These references indicate file descriptors for reading ( *readfds ), for writing ( *writefds ), and those to be checked for exceptions (e.g., message out of band : *exceptfds ). The initial argument, n , is the number of bits in the masks that should be processed. As these masks are 0-based, passing the value 4 indicates the first four bits, representing descriptors 0 to 3, are to be used. The final select argument, *timeout , references a timeval structure that contains information about the length of time the system should wait before completing the select call.

The read, write, and exception file descriptor [15] masks are actually arrays of long integers. On most Linux systems the number of file descriptors, FD_SETSIZE, that can be represented by a mask is 1024 (descriptors 01023). The first bit in the first element of the array is for file descriptor 0, the second bit for file descriptor 1, and so on. If the process does not need to check any descriptors for pending reads, the read descriptor mask may be set to NULL. This also applies for the write and exception masks.

[15] These are file descriptors, not file pointers. When using select , a socket descriptor is treated the same as a file descriptor.

To simplify referencing a specific file descriptor represented by a single bit, several bit manipulation macros are offered . These macros, whose descriptions are usually found on the manual page for select , are

void FD_ZERO(fd_set *fdset);
void FD_SET(int fd, fd_set *fdset);
void FD_CLR(int fd, fd_set *fdset);
int FD_ISSET(int fd, fd_set *fdset);

Each macro must be passed a reference to the address of the file descriptor mask to manipulate. The FD_ZERO macro will zero (set to all zeros) the referenced mask. The FD_SET macro will set the appropriate bit for the passed file descriptor value. The FD_CLEAR macro will clear the bit for the passed file descriptor. The FD_ISSET macro will return, without changing its state, the status of the bit for the passed file descriptor (0 for not set and 1 for set). In practice, the FD_ISSET macro is used when the select call returns to determine which descriptors are actually ready for the indicated I/O event.

The last argument for select specifies the amount of time the call should wait before completing its action and returning. This argument references a timeval structure, which is shown below:

struct timeval {
 long tv_sec; /* seconds */
 long tv_usec; /* and microseconds */
};

If this argument is set to NULL, the select call will wait (block) indefinitely until one of the specified descriptors is ready for I/O. If the tv_sec member and the tv_usec member are both set to 0, the select call will poll the specified descriptors and return immediately with their status. If the timeval members are nonzero, the system will wait the indicated number of seconds/microseconds for an I/O event to occur or return immediately if one of the indicated events occurs prior to the expiration of the specified time. [16] While the Linux version of the select call is fairly standard, it does differ in one very important way. Upon return, the timeout argument for select is modified to reflect the amount of time not slept. Thus, if the call to select is invoked multiple times (or in a loop), the timeout argument must be reinitialized before each call. This difference in behavior should be accounted for when porting code.

[16] If all the file descriptor masks are empty, the time specification for select can be used to fine-tune the amount of time a process should sleep.

If select is successful, it returns the number of ready file descriptors. If the call has timed out, it returns a 0. If the call fails, it returns a -1 and sets errno to one of the values shown in Table 10.26. The file descriptor masks are modified to reflect the current status of the descriptors when the select call is successful or has timed out. The masks are not modified in the event of an error.

Table 10.26. select Error Messages

#	Constant	perror Message	Explanation
4	EINTR	Interrupted system call	A signal was received by process before any of the indicated events occurred or time limit expired .
9	EBADF	Bad file descriptor	One of the file descriptor masks references an invalid file descriptor.
12	ENOMEM	Cannot allocate memory	System unable to allocate internal tables used by select .
22	EINVAL	Invalid argument	One of the time limit values is out of range or file descriptor is negative.

A closing note about select : The first argument, which indicates the number of bits that select will process in the file descriptor mask(s), must be assigned the value of the largest file (socket) descriptor value plus 1 (remember references are 0-based). Finally, select , while still widely used is on the deprecated call hit list in Linux. The system call poll , a variation of select , provides similar functionality.

Program 10.13, which shows how the select call can be used, is a modification of the original Internet domain connectionless server program (10.10). Modified statements and new lines of code are in bold and are placed in gray (lines 5, 911, 3744, 5465, 72, and 73).

Program 10.13 Using select to multiplex I/O in the server program.

File : p10.13.cxx
 /*
 Program 10.13 - SERVER - Internet Domain - connectionless
 */
 #include "local_sock.h"

+ #include 

int
 main( ) {
 int sock, n,

n_ready, need_rsp;


10 fd_set read_fd;


struct timeval w_time;

socklen_t server_len, client_len;
 struct sockaddr_in server, // Internet Addresses
 + client;
 // SOCKET
 if ((sock = socket(PF_INET, SOCK_DGRAM, 0)) < 0) {
 perror("SERVER socket "); return 1;
 }
 20 memset(&server, 0, sizeof(server)); // Clear structure
 server.sin_family = AF_INET; // Set address type
 server.sin_addr.s_addr = htonl(INADDR_ANY);
 server.sin_port = htons(0);
 // BIND
 + if (bind(sock, (struct sockaddr *) &server,
 sizeof(server) ) < 0) {
 perror("SERVER bind "); return 2;
 }
 server_len = sizeof(server); // Obtain address length
 30 // Find picked port #
 if (getsockname(sock, (struct sockaddr *) &server,
 &server_len) < 0) {
 perror("SERVER getsocketname "); return 3;
 }
 + cout << "Server using port " << ntohs(server.sin_port) << endl;
 while ( 1 ) { // Loop forever

w_time.tv_sec = 5; w_time.tv_usec = 0; // set the wait time


FD_ZERO( &read_fd ); // zero all bits


FD_SET( sock, &read_fd ); // indicate one to read


40 if ( (n_ready=select( sock + 1, &read_fd, (fd_set *) NULL,


(fd_set *) NULL, &w_time)) < 0 ) {


perror("SERVER read socket select "); continue;


}


if ( FD_ISSET( sock, &read_fd ) ) { // activity on socket

+ client_len = sizeof(client); // estimate length
 memset(buf, 0, BUFSIZ); // clear the buffer
 if ((n=recvfrom(sock, buf, BUFSIZ, 0, // get the client's msg
 (struct sockaddr *) &client, &client_len)) < 0){
 perror("SERVER recvfrom ");
 50 close(sock); return 4;
 }
 write( fileno(stdout), buf, n ); // display msg on server
 memset(buf, 0, BUFSIZ); // clear the buffer

need_rsp = 1;


+ }


if ( need_rsp ) {


w_time.tv_sec = 5; w_time.tv_usec = 0; // set the wait time


FD_ZERO( &read_fd ); // zero all bits


FD_SET( fileno(stdin), &read_fd ); // the one to read


60 if ( (n_ready=select( fileno(stdin) + 1, &read_fd,


(fd_set *) NULL,(fd_set *) NULL, &w_time)) < 0 ) {


perror("SERVER read stdin select "); continue;


} // get server's msg


// if activity stdin


+ if ( FD_ISSET( fileno(stdin), &read_fd ) ) {

if ( read( fileno(stdin),buf,BUFSIZ) != 0 ) {
 if ((sendto(sock, buf, strlen(buf) ,0, // send to client
 (struct sockaddr *) &client, client_len)) <0){
 perror("SERVER sendto ");
 70 close(sock); return 5;
 }

need_rsp = 0;


}

}
 + }
 }
 return 0;
 }

The modified server program adds the include file , since it makes reference to the timeval structure. Two integer variables have also been added. The n_read variable will be assigned the number of ready I/O descriptors found by the select call. In this setting, this variable should only contain the value 0 or 1. The second variable, need_rsp , is used as a flag to keep track of whether or not the server has responded to a received message. As the address of the process sending the message is included with the message, a sendto call to respond to the message cannot be issued until a message address pair has been received. The mask that represents the descriptors for reading, read_fd , is allocated next . Following this, a structure to hold the time to wait for the select call is allocated.

In the processing loop, the wait time is set arbitrarily to 5 seconds, the read descriptor mask is zeroed, and the bit to indicate the socket to process is set. The select call is used to determine the status of the socket. Since we are only interested in reading, the remaining descriptor masks are set to NULL (note that each is cast appropriately). When the select call returns, the FD_ISSET macro determines if the socket is actually available for reading. If the socket is ready, the message is received, via recvfrom , and displayed. Once the message is displayed, the need_rsp variable is set to 1 to flag the reception .

After checking for received messages and having either received a message or timed out waiting, the server looks to send a response. The need_rsp variable is evaluated to determine if a response to a message should be generated. If a response is needed, the wait time is reset, the read_fd mask is reset to reference stdin , and a call to select is made to determine if any input (in this setting, from the keyboard) is available for reading. If the user running the server process has entered information, it is read and sent to the client. After a message has been sent, the need_rsp variable is set to 0 to indicate a response was generated and sent.

While these changes help the server process to better handle asynchronous communications with the client process, it does not resolve all of the communication problems. The client process must also be changed in a similar manner to allow for non-blocking asynchronous communications. There are additional coordination problems to address. For example, say there are multiple clients and the user running the server is slow in responding. Once the user on the server does respond, how does the server process know (keep track of) to whom it should send the response if, in the interim, it has received additional messages from other clients ?