21.4 Network IO

21.4 Network I/O

Writing to a remote object on the Internet is not very different from writing to a file on your local machine. You might want to do this if your program needs to store its data to a file on a machine on your network, or if you are creating a program that displays information on a monitor connected to another computer on your network.

Network I/O is based on the use of streams created with sockets. Sockets are very useful for client/server applications, peer to peer (P2P), and when making remote procedure calls.

A socket is an object that represents an endpoint for communication between processes communicating across a network. Sockets can work with various protocols, including UDP and TCP/IP. In this section, we create a TCP/IP connection between a server and a client. TCP/IP is a connection-based protocol for network communication. Connection-based means that with TCP/IP, once a connection is made, the two processes can talk with one another as if they were connected by a direct phone line.

Although TCP/IP is designed to talk across a network, you can simulate network communication by running the two processes on the same machine.

It is possible for more than one application on a given computer to be talking to various clients all at the same time (e.g., you might be running a web server, an FTP server, and a program that provides calculation support). Therefore, each application must have a unique ID so that the client can indicate which application it is looking for. That ID is known as a port. Think of the IP address as a phone number and the port as an extension.

The server instantiates a socket and tells that socket to listen for connections on a specific port. The constructor for the socket has one parameter: an int representing the port on which that socket should listen.

Client applications connect to a specific IP address. For example, Yahoo's IP address is 216.115.108.245. Clients must also connect to a specific port. All web browsers connect to port 80 by default. Port numbers range from 0 to 65,535 (i.e., 216); however, some numbers are reserved.

Ports are divided into the following ranges:

• 0-1023: well-known ports

• 1024-49151: registered ports

• 49152-65535: dynamic and/or private ports

For a list of all the well-known and registered ports, look at http://www.iana.org/assignments/port-numbers.

If you are running your program on a network with a firewall, talk to your network administrator about which ports are closed.

Once the socket is created, call Start( ) on it, telling the socket to begin accepting network connections. When the server is ready to start responding to calls from clients, call AcceptSocket( ). The thread in which you've called AcceptSocket( ) blocks (waiting sadly by the phone, wringing its virtual hands, hoping for a call).

You can imagine creating the world's simplest socket. It waits patiently for a client to call. When it gets a call, it interacts with that client to the exclusion of all other clients. The next few clients to call will connect, but they will automatically be put on hold. While they are listening to the music and being told their call is important and will be handled in the order received, they will block in their own threads. Once the backlog (hold) queue fills, subsequent callers will get the equivalent of a busy signal. They must hang up and wait for our simple socket to finish with its current client. This model works fine for servers that take only one or two requests a week, but it doesn't scale well for real-world applications. Most servers need to handle thousands, even tens of thousands of connections a minute!

To handle a high volume of connections, applications use asynchronous I/O to accept a call and return a new socket with the connection to the client. The original socket then returns to listening, waiting for the next client. This way your application can handle many calls; each time a call is accepted, a new socket is created.

The client is unaware of this sleight of hand in which a new socket is created. As far as the client is concerned, he has connected with the socket at the IP address and port he requested. Note that the new socket establishes a persistent connection with the client. This is quite different from UDP, which uses a connectionless protocol. With TCP/IP, once the connection is made, the client and server know how to talk with each other without having to readdress each packet.

The Socket class itself is fairly simple. It knows how to be an endpoint, but it doesn't know how to accept a call and create a TCP/IP connection. This is actually done by the TcpListener class. The TcpListener class builds upon the Socket class to provide high-level TCP/IP services.

21.4.1 Creating a Network Streaming Server

To create a network server for TCP/IP streaming, start by creating a TcpListener object to listen to the TCP/IP port you've chosen. I've arbitrarily chosen port 65000 from the available port IDs:

IPAddress localAddr = IPAddress.Parse("127.0.0.1"); TcpListener tcpListener = new TcpListener(localAddr, 65000);

Once the TcpListener object is constructed, you can ask it to start listening:

tcpListener.Start( );  

Now wait for a client to request a connection:

Socket socketForClient = tcpListener.AcceptSocket( );

The AcceptSocket method of the TcpListener object returns a Socket object that represents a Berkeley socket interface and is bound to a specific endpoint. AcceptSocket( ) is a synchronous method that will not return until it receives a connection request.

Because the model is widely accepted by computer vendors, Berkeley sockets simplify the task of porting existing socket-based source code from both Windows and Unix environments.

If the socket is connected, you're ready to send the file to the client:

if (socketForClient.Connected) {

Create a NetworkStream class, passing the socket into the constructor:

NetworkStream networkStream = new NetworkStream(socketForClient);

Then create a StreamWriter object much as you did before, except this time not on a file, but rather on the NetworkStream you just created:

System.IO.StreamWriter streamWriter = new System.IO.StreamWriter(networkStream);

When you write to this stream, the stream is sent over the network to the client. Example 21-8 shows the entire server. (I've stripped this server down to its bare essentials. With a production server, you almost certainly would run the request processing code in a thread, and you'd want to enclose the logic in try blocks to handle network problems.)

Example 21-8. Implementing a network streaming server

using System; using System.Net; using System.Net.Sockets; public class NetworkIOServer {       public static void Main( )    {       NetworkIOServer app =           new NetworkIOServer( );       app.Run( );    }    private void Run( )    {       // create a new TcpListener and start it up       // listening on port 65000       IPAddress localAddr = IPAddress.Parse("127.0.0.1");       TcpListener tcpListener = new TcpListener(localAddr, 65000);       tcpListener.Start( );              // keep listening until you send the file       for (;;)       {          // if a client connects, accept the connection          // and return a new socket named socketForClient          // while tcpListener keeps listening          Socket socketForClient =              tcpListener.AcceptSocket( );          if (socketForClient.Connected)          {             Console.WriteLine("Client connected");             // call the helper method to send the file             SendFileToClient(socketForClient);             Console.WriteLine(                "Disconnecting from client...");             // clean up and go home             socketForClient.Close( );             Console.WriteLine("Exiting...");             break;          }       }    }    // helper method to send the file    private void SendFileToClient(       Socket socketForClient )    {       // create a network stream and a stream writer        // on that network stream       NetworkStream networkStream =           new NetworkStream(socketForClient);       System.IO.StreamWriter streamWriter =           new System.IO.StreamWriter(networkStream);       // create a stream reader for the file       System.IO.StreamReader streamReader =           new System.IO.StreamReader(             @"C:\test\source\myTest.txt");       string theString;       // iterate through the file, sending it        // line-by-line to the client       do       {          theString = streamReader.ReadLine( );          if( theString != null )          {             Console.WriteLine(                "Sending {0}", theString);             streamWriter.WriteLine(theString);             streamWriter.Flush( );          }       }       while( theString != null );       // tidy up       streamReader.Close( );       networkStream.Close( );       streamWriter.Close( );    } }

21.4.2 Creating a Streaming Network Client

The client instantiates a TcpClient class, which represents a TCP/IP client connection to a host:

TcpClient socketForServer; socketForServer = new TcpClient("localHost", 65000);

With this TcpClient, you can create a NetworkStream, and on that stream you can create a StreamReader:

NetworkStream networkStream = socketForServer.GetStream( ); System.IO.StreamReader streamReader =      new System.IO.StreamReader(networkStream);

Now read the stream as long as there is data on it, outputting the results to the console:

do {     outputString = streamReader.ReadLine( );     if( outputString != null )     {         Console.WriteLine(outputString);     } } while( outputString != null );

Example 21-9 is the complete client.

Example 21-9. Implementing a network streaming client

using System; using System.Net.Sockets; public class Client {         static public void Main( string[] Args )    {       // create a TcpClient to talk to the server       TcpClient socketForServer;       try       {          socketForServer =              new TcpClient("localHost", 65000);       }       catch       {          Console.WriteLine(             "Failed to connect to server at {0}:65000",                 "localhost");          return;       }       // create the Network Stream and the Stream Reader object       NetworkStream networkStream =              socketForServer.GetStream( );       System.IO.StreamReader streamReader =           new System.IO.StreamReader(networkStream);                try       {          string outputString;          // read the data from the host and display it          do          {             outputString = streamReader.ReadLine( );             if( outputString != null )             {                Console.WriteLine(outputString);             }          }          while( outputString != null );       }       catch       {          Console.WriteLine(             "Exception reading from Server");       }       // tidy up        networkStream.Close( );    } }

To test this, I created a simple test file named myText.txt:

This is line one This is line two This is line three This is line four

Here is the output from the server and the client:

Output (Server): Client connected Sending This is line one Sending This is line two Sending This is line three Sending This is line four Disconnecting from client... Exiting... Output (Client): This is line one This is line two This is line three This is line four Press any key to continue

If you are testing this on a single machine, run the client and server in separate command windows or individual instances of the development environment. You will want to start the server first or the client will fail, saying it could not connect.

21.4.3 Handling Multiple Connections

As mentioned earlier, this example does not scale well. Each client demands the entire attention of the server. A server is needed that can accept the connection and then pass the connection to overlapped I/O, providing the same asynchronous solution that you used earlier for reading from a file.

To manage this, create a new server, AsynchNetworkServer, which will nest within it a new class, ClientHandler. When your AsynchNetworkServer receives a client connection, it will instantiate a ClientHandler and pass the socket to that ClientHandler instance.

The ClientHandler constructor will create a copy of the socket and a buffer and will open a new NetworkStream on that socket. It will then use overlapped I/O to asynchronously read and write to that socket. For this demonstration, it will simply echo whatever text the client sends, back to the client and also to the console.

To create the asynchronous I/O, ClientHandler will define two delegate methods, OnReadComplete( ) and OnWriteComplete( ), that will manage the overlapped I/O of the strings sent by the client.

The body of the Run( ) method for the server is very similar to what you saw in Example 21-8. First, create a listener and then call Start( ). Then create a forever loop and call AcceptSocket( ). Once the socket is connected, rather than handling the connection, create a new ClientHandler and call StartRead( ) on that object.

The complete source for the server is shown in Example 21-10.

Example 21-10. Implementing an asynchronous network streaming server

using System; using System.Net; using System.Net.Sockets; public class AsynchNetworkServer {       class ClientHandler    {       private byte[]          buffer;       private Socket          socket;       private NetworkStream   networkStream;       private AsyncCallback   callbackRead;       private AsyncCallback   callbackWrite;       public ClientHandler( Socket socketForClient )       {          socket = socketForClient;          buffer = new byte[256];          networkStream =              new NetworkStream(socketForClient);          callbackRead =             new AsyncCallback(this.OnReadComplete);          callbackWrite =             new AsyncCallback(this.OnWriteComplete);       }       // begin reading the string from the client       public void StartRead( )       {          networkStream.BeginRead(             buffer, 0, buffer.Length,              callbackRead, null);       }       // when called back by the read, display the string       // and echo it back to the client       private void OnReadComplete( IAsyncResult ar )       {          int bytesRead = networkStream.EndRead(ar);          if( bytesRead > 0 )          {             string s =                 System.Text.Encoding.ASCII.GetString(                   buffer, 0, bytesRead);             Console.Write(                    "Received {0} bytes from client: {1}",                     bytesRead, s );             networkStream.BeginWrite(                buffer, 0, bytesRead, callbackWrite, null);          }          else          {             Console.WriteLine( "Read connection dropped");             networkStream.Close( );             socket.Close( );             networkStream = null;             socket = null;          }       }       // after writing the string, print a message and resume reading       private void OnWriteComplete( IAsyncResult ar )       {          networkStream.EndWrite(ar);          Console.WriteLine( "Write complete");          networkStream.BeginRead(             buffer, 0, buffer.Length,              callbackRead, null);       }    }    public static void Main( )    {       AsynchNetworkServer app =           new AsynchNetworkServer( );       app.Run( );    }    private void Run( )    {       // create a new TcpListener and start it up       // listening on port 65000       IPAddress localAddr = IPAddress.Parse("127.0.0.1");       TcpListener tcpListener = new TcpListener(localAddr, 65000);       tcpListener.Start( );              // keep listening until you send the file       for (;;)       {          // if a client connects, accept the connection          // and return a new socket named socketForClient          // while tcpListener keeps listening          Socket socketForClient =              tcpListener.AcceptSocket( );          if (socketForClient.Connected)          {             Console.WriteLine("Client connected");             ClientHandler handler =                 new ClientHandler(socketForClient);             handler.StartRead( );         }       }    } }

The server starts up and listens to port 65000. If a client connects, the server will instantiate a ClientHandler that will manage the I/O with the client while the server listens for the next client.

In this example, write the string received from the client to the console in OnReadComplete( ) and OnWriteComplete( ). Writing to the console can block your thread until the write completes. In a production program, you do not want to take any blocking action in these methods, because you are using a pooled thread. If you block in OnReadComplete( ) or OnWriteComplete( ), you may cause more threads to be added to the thread pool, which is inefficient and will harm performance and scalability.

The client code is very simple. The client creates a tcpSocket for the port on which the server will listen (65000) and creates a NetworkStream object for that socket. It then writes a message to that stream and flushes the buffer. The client creates a StreamReader to read on that stream and writes whatever it receives to the console. The complete source for the client is shown in Example 21-11.

Example 21-11. Implementing a client for asynchronous network I/O

using System; using System.Net.Sockets; using System.Threading; using System.Runtime.Serialization.Formatters.Binary; public class AsynchNetworkClient {    private NetworkStream   streamToServer;    static public int Main( )    {       AsynchNetworkClient client =           new AsynchNetworkClient( );       return client.Run( );    }    AsynchNetworkClient( )    {       string serverName = "localhost";       Console.WriteLine("Connecting to {0}", serverName);       TcpClient tcpSocket = new TcpClient(serverName, 65000);       streamToServer = tcpSocket.GetStream( );    }    private int Run( )    {       string message = "Hello Programming C#";       Console.WriteLine(           "Sending {0} to server.", message);       // create a streamWriter and use it to       // write a string to the server       System.IO.StreamWriter writer =           new System.IO.StreamWriter(streamToServer);       writer.WriteLine(message);       writer.Flush( );       // Read response       System.IO.StreamReader reader =           new System.IO.StreamReader(streamToServer);       string strResponse = reader.ReadLine( );       Console.WriteLine("Received: {0}", strResponse);       streamToServer.Close( );       return 0;    } } Output (Server): Client connected Received 22 bytes from client: Hello Programming C# Write complete Read connection dropped Output (Client): Connecting to localhost Sending Hello Programming C# to server. Received: Hello Programming C#

In this example, the network server does not block while it is handling client connections, but rather it delegates the management of those connections to instances of ClientHandler. Clients should not experience a delay waiting for the server to handle their connections.

21.4.4 Asynchronous Network File Streaming

You can now combine the skills learned for asynchronous file reads with asynchronous network streaming, to produce a program that serves a file to a client on demand.

Your server will begin with an asynchronous read on the socket, waiting to get a filename from the client. Once you have the filename, you can kick off an asynchronous read of that file on the server. As each buffer-ful of the file becomes available, you can begin an asynchronous write back to the client. When the asynchronous write to the client finishes, you can kick off another read of the file; in this way you ping-pong back and forth, filling the buffer from the file and writing the buffer out to the client. The client need do nothing but read the stream from the server. In the next example, the client will write the contents of the file to the console, but you could easily begin an asynchronous write to a new file on the client, thereby creating a network-based file copy program.

The structure of the server is not unlike that shown in Example 21-10. Once again you will create a ClientHandler class, but this time add an AsyncCallBack named myFileCallBack, which you initialize in the constructor along with the callbacks for the network read and write.

myFileCallBack =     new AsyncCallback(this.OnFileCompletedRead); callbackRead =    new AsyncCallback(this.OnReadComplete); callbackWrite =    new AsyncCallback(this.OnWriteComplete);

The Run( ) function of the outer class, now named AsynchNetworkFileServer, is unchanged. Once again you create and start the TcpListener class as well as create a forever loop in which you call AcceptSocket( ). If you have a socket, instantiate the ClientHandler and call StartRead( ). As in the previous example, StartRead( ) kicks off a BeginRead( ), passing in the buffer and the delegate to OnReadComplete.

When the read from the network stream completes, your delegated method OnReadComplete( ) is called and it retrieves the filename from the buffer. If text is returned, OnReadComplete( ) retrieves a string from the buffer using the static System.Text.Encoding.ASCII.GetString( ) method:

if( bytesRead > 0 ) {    string fileName =        System.Text.Encoding.ASCII.GetString(       buffer, 0, bytesRead);

You now have a filename; with that, you can open a stream to the file and use the exact same asynchronous file read used in Example 21-7.

inputStream =     File.OpenRead(fileName); inputStream.BeginRead(    buffer,             // holds the results    0,                  // offset    buffer.Length,      // Buffer Size    myFileCallBack,     // call back delegate    null);              // local state object

This read of the file has its own callback that will be invoked when the input stream has read a buffer-ful from the file on the server disk drive.

As noted earlier, you normally would not want to take any action in an overlapped I/O method that might block the thread for any appreciable time. The call to open the file and begin reading it would normally be pushed off to a helper thread, instead of doing this work in OnReadComplete( ). It has been simplified for this example to avoid distracting from the issues at hand.

When the buffer is full, OnFileCompletedRead( ) is called, which checks to see if any bytes were read from the file. If so, it begins an asynchronous write to the network:

if (bytesRead > 0) {    // write it out to the client    networkStream.BeginWrite(       buffer, 0, bytesRead, callbackWrite, null); }

If OnFileCompletedRead was called and no bytes were read, this signifies that the entire file has been sent. The server reacts by closing the NetworkStream and socket, thus letting the client know that the transaction is complete:

networkStream.Close( ); socket.Close( ); networkStream = null; socket = null;

When the network write completes, the OnWriteComplete( ) method is called, and this kicks off another read from the file:

private void OnWriteComplete( IAsyncResult ar ) {    networkStream.EndWrite(ar);    Console.WriteLine( "Write complete");    inputStream.BeginRead(       buffer,             // holds the results       0,                  // offset       buffer.Length,      // (BufferSize)       myFileCallBack,         // call back delegate       null);              // local state object }

The cycle begins again with another read of the file, and the cycle continues until the file has been completely read and transmitted to the client. The client code simply writes a filename to the network stream to kick off the file read:

string message = @"C:\test\source\AskTim.txt"; System.IO.StreamWriter writer =      new System.IO.StreamWriter(streamToServer);  writer.Write(message);  writer.Flush( );

The client then begins a loop, reading from the network stream until no bytes are sent by the server. When the server is done, the network stream is closed. Start by initializing a Boolean value to false and creating a buffer to hold the bytes sent by the server:

bool fQuit = false; while (!fQuit) {    char[] buffer = new char[BufferSize];

You are now ready to create a new StreamReader from the NetworkStream member variable streamToServer:

System.IO.StreamReader reader =     new System.IO.StreamReader(streamToServer);

The call to Read( ) takes three parameters: the buffer, the offset at which to begin reading, and the size of the buffer.

int bytesRead = reader.Read(buffer,0, BufferSize);

Check to see if the Read( ) returned any bytes; if not, you are done and you can set the Boolean value fQuit to true, causing the loop to terminate:

if (bytesRead == 0)    fQuit = true;

If you did receive bytes, you can write them to the console, or write them to a file, or do whatever it is you will do with the values sent from the server:

   else    {       string theString = new String(buffer);       Console.WriteLine(theString);    } }

Once you break out of the loop, close the NetworkStream.

streamToServer.Close( );

The complete annotated source for the server is shown in Example 21-12, with the client following in Example 21-13.

Example 21-12. Implementing an asynchronous network file server

using System; using System.Net; using System.Net.Sockets; using System.Text; using System.IO; // get a file name from the client // open the file and send the // contents from the server to the client public class AsynchNetworkFileServer {       class ClientHandler    {       private const int       BufferSize = 256;       private byte[]          buffer;       private Socket          socket;       private NetworkStream   networkStream;       private Stream          inputStream;       private AsyncCallback   callbackRead;       private AsyncCallback   callbackWrite;       private AsyncCallback   myFileCallBack;       // constructor       public ClientHandler(           Socket socketForClient )       {          // initialize member variable          socket = socketForClient;          // initialize buffer to hold          // contents of file          buffer = new byte[256];          // create the network stream          networkStream =              new NetworkStream(socketForClient);          // set the file callback for reading          // the file          myFileCallBack =              new AsyncCallback(this.OnFileCompletedRead);          // set the callback for reading from the           // network stream          callbackRead =             new AsyncCallback(this.OnReadComplete);          // set the callback for writing to the          // network stream          callbackWrite =             new AsyncCallback(this.OnWriteComplete);       }       // begin reading the string from the client       public void StartRead( )       {          // read from the network          // get a filename          networkStream.BeginRead(             buffer, 0, buffer.Length,              callbackRead, null);       }       // when called back by the read, display the string       // and echo it back to the client       private void OnReadComplete( IAsyncResult ar )       {          int bytesRead = networkStream.EndRead(ar);          // if you got a string          if( bytesRead > 0 )          {             // turn the string to a file name             string fileName =                 System.Text.Encoding.ASCII.GetString(                buffer, 0, bytesRead);             // update the console             Console.Write(                 "Opening file {0}", fileName);             // open the file input stream             inputStream =                 File.OpenRead(fileName);             // begin reading the file             inputStream.BeginRead(                buffer,             // holds the results                0,                  // offset                buffer.Length,      // BufferSize                myFileCallBack,     // call back delegate                null);              // local state object          }          else          {             Console.WriteLine( "Read connection dropped");             networkStream.Close( );             socket.Close( );             networkStream = null;             socket = null;          }       }       // when you have a buffer-full of the file       void OnFileCompletedRead(IAsyncResult asyncResult)       {          int bytesRead =              inputStream.EndRead(asyncResult);          // if you read some file          if (bytesRead > 0)          {             // write it out to the client             networkStream.BeginWrite(                buffer, 0, bytesRead, callbackWrite, null);          }          else          {             Console.WriteLine("Finished.");             networkStream.Close( );             socket.Close( );             networkStream = null;             socket = null;          }       }       // after writing the string, get more of the file       private void OnWriteComplete( IAsyncResult ar )       {          networkStream.EndWrite(ar);          Console.WriteLine( "Write complete");          // begin reading more of the file          inputStream.BeginRead(             buffer,             // holds the results             0,                  // offset             buffer.Length,      // (BufferSize)             myFileCallBack,         // call back delegate             null);              // local state object       }    }    public static void Main( )    {       AsynchNetworkFileServer app =           new AsynchNetworkFileServer( );       app.Run( );    }    private void Run( )    {       // create a new TcpListener and start it up       // listening on port 65000       IPAddress localAddr = IPAddress.Parse("127.0.0.1");       TcpListener tcpListener = new TcpListener(localAddr, 65000);       tcpListener.Start( );              // keep listening until you send the file       for (;;)       {          // if a client connects, accept the connection          // and return a new socket named socketForClient          // while tcpListener keeps listening          Socket socketForClient =              tcpListener.AcceptSocket( );          if (socketForClient.Connected)          {             Console.WriteLine("Client connected");             ClientHandler handler =                 new ClientHandler(socketForClient);             handler.StartRead( );          }       }    } }

Example 21-13. Implementing a client for an asynchronous network file server

using System; using System.Net.Sockets; using System.Threading; using System.Text; public class AsynchNetworkClient {    private const int BufferSize = 256;    private NetworkStream   streamToServer;    static public int Main( )    {       AsynchNetworkClient client =           new AsynchNetworkClient( );       return client.Run( );    }    AsynchNetworkClient( )    {       string serverName = "localhost";       Console.WriteLine("Connecting to {0}", serverName);       TcpClient tcpSocket = new TcpClient(serverName, 65000);       streamToServer = tcpSocket.GetStream( );    }    private int Run( )    {       string message = @"C:\test\source\AskTim.txt";       Console.Write(          "Sending {0} to server.", message);       // create a streamWriter and use it to       // write a string to the server       System.IO.StreamWriter writer =           new System.IO.StreamWriter(streamToServer);       writer.Write(message);       writer.Flush( );       bool fQuit = false;       // while there is data coming       // from the server, keep reading       while (!fQuit)       {          // buffer to hold the response          char[] buffer = new char[BufferSize];          // Read response          System.IO.StreamReader reader =              new System.IO.StreamReader(streamToServer);          // see how many bytes are           // retrieved to the buffer          int bytesRead =              reader.Read(buffer,0,BufferSize);          if (bytesRead == 0)  // none? quite             fQuit = true;          else                 // got some?          {             // display it as a string             string theString = new String(buffer);             Console.WriteLine(theString);          }       }       streamToServer.Close( ); // tidy up       return 0;    } }

By combining the asynchronous file read with the asynchronous network read, you have created a scalable application that can handle requests from a number of clients.