Networking

Programs don't live in isolation. Interprogram and interprocess communication on a single machine can be used to connect software components, for example using the .NET Remoting technology. But once you need to step off the local machine, a new set of APIs and concepts is required to communicate over the ubiquitous network. The Networking Class Libraries (NCL) in the System.Net namespaces exist to enable those scenarios.

We'll see in this section how to perform network operations through a variety of classes and common protocols, including sockets, and protocols like TCP/IP, UDP, HTTP, FTP, and SMTP. Each has dedicated library support. While an in-depth discussion of each of these is beyond the scope of this book, we provide a brief discussion of each before describing the .NET Framework's support. As is always the case, the "Further Reading" section offers some resources with deeper coverage of these technologies.

Many business applications are better suited staying at a higher level of abstraction rather than communicating at the wire level. These protocols are typically used for low-level systems, such as utilities and reusable libraries, server programs, and network-focused client applications. Please refer the "Further Reading" section for resources on .NET Remoting, Web Services, and Native Messaging (e.g., MSMQ), each of which can be used for higher-level RPC and messaging orchestration.

Sockets

A standard interface for low-level network programming was created in 1983 as part of the Berkeley Software Distribution (BSD) OS created at the University of California, Berkeley. This technology was called the Berkeley Sockets API, now commonly referred to just as sockets. There are many interoperable implementations of this API today on various OSs.

A socket represents one end of a connection between two end points on the network (much like a stream). Windows ships an implementation of sockets called WinSock32 (or just WinSock), on top of which the .NET Framework builds the System.Net.Sockets APIs. The library can be used for both responding to and initiating new network communication, which makes them perfect for both server- or client-based networked applications. They are often used for work-intensive server applications — such as web servers, for example — that must continually listen for and respond to incoming network requests. While sockets are typically more complex to work with than some of the other protocol-specific client and listener types — discussed further below — they are very powerful and versatile, enabling you to work with the raw bits and message exchange protocols in whatever manner you choose. It's safe to conjecture that most of today's client-server applications are built on top of some implementation of sockets.

A socket is a single end point to which data can be written and from which data can be read. Because of sockets' similarity to streams, you can even abstract a socket by wrapping it in a NetworkStream and using it like you would any other stream type. The functionality we discuss below is quite comparable to the familiar idioms discussed earlier in this section. Where it differs is mostly limited to the details around binding and connecting to network end points.

Sockets Terminology

Sockets are standardized. With this standardization comes a bit of standard terminology — one of the benefits of such an ubiquitous technology. The Socket implementation in the System.Net.Sockets namespace is no different. The primary terms or operations — described further in the coming sections — are socket, bind, listen, accept, connect, send, receive, and close. Each has a corresponding method on the Socket type. In brief, abstract terms, these mean the following:

• Socket: This operation is used to create a new socket end point, allocating underlying OS resources that can then be used to execute incoming and outgoing network communication. The Socket constructor takes the place of the socket operation.

• Bind: Servers must bind sockets to an address in order to establish a local name. Clients do not use this operation. A bind enables external communication to connect to and send messages through the new end point, and enables the socket to read messages off of it and/or send its own. A local name includes the network port on which to listen, where the port is simply a number to which incoming requests can direct their communication.

• Listen: To enable queuing of incoming requests for consumption, a server-side socket must explicitly indicate the size of its queue. By executing a listen operation, a socket notifies the OS that it is interested in participating in queuing. Although executing a listen is optional, without doing so any incoming requests will be rejected immediately if a socket application is already processing a message.

• Accept: A socket accepts an incoming request by forking off a new socket, which can be used to read and write to the client, enabling the original server socket to go back and accept additional incoming messages. Accept blocks until a request is available, either as it gets sent by a client or by pulling it off of the listener queue.

• Connect: Clients use this operation to connect to a server-side network end point (server in this case is any network end point listening for incoming messages). Once the target end point accepts the request, the socket can be used to read from and write data to the socket.

• Send: Once a socket, whether server- or client-based, is hooked up to an end point, sending data initiates a message that the other end point is then able to receive;

• Receive: After the end point to which a socket is connected sends data, a socket may consume that data by receiving it. If no data is available, this operation blocks until data becomes available.

• Close: When a connection has finished its session, a socket must be closed. This relinquishes any resources held by the socket, and, if bound, disassociates the end point.

We will spend the following sections discussing how to perform these operations and the arguments they require. We will also break the discussion up into common functionality and that specific to server and client-based sockets, as the messaging patterns are sufficiently different to warrant doing so. Note that in the case of Peer-to-Peer (P2P) programs, a single program can act as both the server and client simultaneously.

Creating the Socket

The first step in working with sockets is to create a new Socket instance. As you may have guessed, this is done through the Socket constructor, of which there is only one overload. The constructor takes three arguments, each of which is an enumeration value of type AddressFamily, SocketType, and ProtocolType. We'll take a look at the possible values for each below. Because of the sheer number of some of the enumeration values, we won't discuss details of each — please refer to the SDK for specific details on a particular address family or protocol type.

AddressFamily specifies the following values:

 public enum AddressFamily {       AppleTalk = 0x10,       Atm = 0x16,       Banyan = 0x15,       Ccitt = 10,       Chaos = 5,       Cluster = 0x18,       DataKit = 9,       DataLink = 13,       DecNet = 12,       Ecma = 8,       FireFox = 0x13,       HyperChannel = 15,       Ieee12844 = 0x19,       ImpLink = 3,       InterNetwork = 2,       InterNetworkV6 = 0x17,       Ipx = 6,       Irda = 0x1a,       Iso = 7,       Lat = 14,       Max = 0x1d,       NetBios = 0x11,       NetworkDesigners = 0x1c,       NS = 6,       Osi = 7,       Pup = 4,       Sna = 11,       Unix = 1,       Unknown = -1,       Unspecified = 0,       VoiceView = 0x12 } 

For most mainstream socket programming you will want to work with the InterNetwork family setting (for IPv4 or alternatively InterNetworkV6 for IPv6). If you need one of the other values, chances are you'll know what it is. Your OS version must explicitly support the family; otherwise, an exception will be generated during construction.

The SocketType value indicates the communication style you intend to participate in. There are five possible values, each of which is briefly described below. We will only discuss using stream-based sockets in this chapter — the others are left to the reader to experiment with and investigate:

• Stream: This is the most common type of socket, and likely the only style you'll ever need to work with. A stream socket permits bidirectional communication, guarantees in-order delivery, and uses a multi-message connection-based session. This type of socket uses TCP as its underlying transport.

• Rdm: Reliable data messaging (RDM) is much like stream-based sockets, the primary difference being support for multiple clients and notification of delivery.

• Seqpacket: Sequential packet-style sockets are identical to stream-based sockets, except that reads discard bytes beyond a specified size. For example, if 2048 bytes are available, yet only1024 are read; the remaining 1024 are implicitly discarded. This is useful when dealing with incoming messages of a fixed and well-known packet size.

• Dgram: A datagram socket can be used for bidirectional, unreliable messages. There is no guarantee that messages will arrive in order, that only one copy of the message will be received, or that they will ever arrive at all.

• Raw: Sockets opened as raw also enable datagram-style messaging and provide access to the underlying transport protocol. This enables you to work at a very low level with protocols like ICMP and does not assume any responsibility for transmitting IP headers. It does preserve raw transmitted data (including headers).

Lastly, the ProtocolType enumeration offers the following values:

 public enum ProtocolType {       Ggp = 3,       Icmp = 1,       IcmpV6 = 0x3a,       Idp = 0x16,       Igmp = 2,       IP = 0,       IPSecAuthenticationHeader = 0x33,       IPSecEncapsulatingSecurityPayload = 50,       IPv4 = 4,       IPv6 = 0x29,       IPv6DestinationOptions = 60,       IPv6FragmentHeader = 0x2c,       IPv6HopByHopOptions = 0,       IPv6NoNextHeader = 0x3b,       IPv6RoutingHeader = 0x2b,       Ipx = 0x3e8,       ND = 0x4d,       Pup = 12,       Raw = 0xff,       Spx = 0x4e8,       SpxII = 0x4e9,       Tcp = 6,       Udp = 0x11,       Unknown = -1,       Unspecified = 0 } 

Tcp is the most common choice for our purposes. For example, when working with a stream-style socket that communicates using TCP/IP over IPv4 (probably the most ubiquitous combination), the following code snippet shows the correct way to create the socket:

 Socket s = new Socket(AddressFamily.InterNetwork,     SocketType.Stream, ProtocolType.Tcp); 

Of course, there are 4650 permutations of these three settings as of 2.0, most of which are invalid or just plain silly.

Sending Data

Once you have a connected Socket, you may send data to the recipient. The Send and BeginSend/EndSend methods offer a synchronous and asynchronous ways to do so, respectively:

 using (Socket s = /*...*/) {     byte[] buffer = /*...*/     int bytesRead = s.Send(buffer);     if (bytesRead != buffer.Length)     {         // The transport buffer was probably full...     }     s.Close(); } 

A send operation can block if the transport buffer is not large enough to hold additional data. You can choose to use nonblocking sends by setting the Socket's Blocking property to false. But doing so can cause the returned number of bytes from Send to be less than the size of the byte[] you attempted to send. In other words, if you aren't careful nonblocking send operations can lose data.

You can similarly use the Write methods on NetworkStream (or form a StreamWriter on top of it and use its methods, too). When you construct a new NetworkStream, some of its constructors accept a bool parameter ownsSocket to indicate whether the stream should close the underlying socket when you are done using it:

 using (Socket s = /*...*/) {     NetworkStream ns = new NetworkStream(s);     StreamWriter sw = new StreamWriter(ns, Encoding.UTF8);     sw.WriteLine("..."); // Writes UTF-8 encoded data to the stream.     // ... } 

You can restrict the legal operations on the NetworkStream by supplying a FileAccess enumeration value to one of the constructor overloads. This can be used to hand out read-only or write-only streams, for example.

Receiving Data

Receiving is similar to sending: you can use the Receive or BeginReceive/EndReceive methods to perform either synchronous or asynchronous operations, respectively. A receive operation blocks until there is data available to consume. If the Block property has been set to false, an exception is thrown to indicate when no data is available:

 using (Socket s = /*...*/) {     byte[] buffer = new byte[1024];     int bytesRead = s.Receive(buffer);     for (int i = 0; i < bytesRead; i++)     {         Console.Write("(#{0}={1:X}) ", i, buffer[i]);     } } 

As with writing data to a socket, a NetworkStream can be used to Read from a socket. Similarly, you can form a StreamReader over it to make encoding and decoding textual data simpler.

Closing the Socket

A socket uses OS resources and thus must be closed once you are done using it. The Socket class uses a pattern similar to Stream: it offers both a Close and Dispose method. Both do the same thing. If you are using C#, it's recommended that you wrap up use of any Socket inside a using statement.

Server-Side Sockets

A server-side application whose purpose is to process network requests commonly sits inside a loop and rapidly consumes and dispatches such requests. The basic pattern is to create, bind, listen, and then continuously accept until the program is shut down. A web server is a great example of such a program, whose sole purpose in life is to receive and process incoming requests as fast as possible. Sockets are the de facto standard for implementing these styles of applications. In this section, we take a look at how to use the types in the System.Net.Sockets namespace to create such programs.

Binding to an End Point

Once a server-side socket has been constructed, it must execute a bind instruction. This actually creates a connection between your Socket instance and a concrete local end point on the machine. The bound address is represented by and provided as an argument to the Bind method as an EndPoint instance.

This type is an abstract class, the only concrete implementation of which is IPEndPoint. Although IPEndPoint can be used to refer to any computer using an IPv4 or IPv6 network address, bind accepts only local addresses:

 using (Socket s = /*...*/) {     s.Bind(new IPEndPoint(IPAddress.Loopback, 9999));     // Now the socket is ready to listen or accept... } 

This shows how to bind to port 9999 on the local machine. A loopback is the common terminology for the localhost, represented both in Windows and various UNIX flavors as the static IPv4 address 127.0.0.1. For IPv6, the loopback is represented by 0:0:0:0:0:0:0:1 (or optionally ::127.0.0.1). They can be retrieved using the static IPAddress properties Loopback and IPv6Loopback, respectively.

Listening for Connections

As noted above, a Socket does not inherently support simultaneous requests. Making a call to Listen sets up a queue that enables buffering pending connection requests (specified by the parameter backlog). If a client request is already being processed and the server has not begun accepting new requests, incoming messages will be enqueued in this buffer. The next accept issued will first check the queue and, if it has a new connection, will dequeue and process the next request. Note that it is still important to consume incoming requests in a timely fashion. Not only is the buffer artificially limited to a maximum of 5, but requests sitting in the buffer for too long will likely time out.

Accepting a Connection

Once a socket is set up and running on a concrete end point, its next order of business is to begin processing incoming connections. Invoking the Accept method blocks until an incoming connection is available, either because one is already in the buffer queue or because a new connection arrives. When a request becomes available for processing, Accept unblocks and returns a new Socket instance. This new socket is identical to the original except that it is assigned a new end point through which the client and server can communicate using reads and writes.

This design allows the server program to immediately resume processing of incoming requests. Often the newly accepted socket is placed on another thread, which handles responding to and accommodating the client's requests while the server asynchronously accepts any new client requests. This is required to ensure that incoming requests do not get rejected because of insufficient queue size:

 using (Socket s = /*...*/) {     while (true)     {         Socket client = s.Accept();         ThreadPool.QueueUserWorkItem(             delegate             {                 // The incoming socket connection is captured as ‘client'.                 // We can perform our work here...             }         );     } } 

With this approach, you need to ensure that your ThreadPool has sufficient worker threads so that calls to QueueUserWorkItem can succeed quickly without having to wait for a new thread to be freed up. Refer to Chapter 11 for details.

As with most blocking operations on the .NET Framework, there is an asynchronous version of the Accept method. BeginAccept follows the APM discussed earlier. Because of its similarity to already discussed patterns, we won't go into further details here.

Example: A File-Server Loop

Of course, there are infinite numbers of ways to design a networked server program, but there are surprisingly few patterns that have arisen over the years. Listing 7-4 demonstrates a simple loop that is representative of the typical server application. All this particular application does is accept requests for a filename, to which the server responds by reading and piping its contents to the client. This is quite similar to what a typical web server would do (massively simplified of course). Security is not even taken into account in this example, something that you'd undoubtedly want to put a great deal of thought into.

Listing 7-4: A simple socket-based file server

 void ServerLoop() {     // Create our socket and bind to port 9999, allowing a queue of 3 requests.     using (Socket s = new Socket(AddressFamily.InterNetwork,         SocketType.Stream, ProtocolType.Tcp))     {         s.Bind(new IPEndPoint(IPAddress.Loopback, 9999));         s.Listen(3);         // Our top-level processing loop...         while (true)         {             // Accept the next client and dispatch to our worker:             Socket client = s.Accept();             ThreadPool.QueueUserWorkItem(HandleRequest, client);         }     } } void HandleRequest(object state) {     // Set up our objects for communication:     using (Socket client = (Socket)state)     using (NetworkStream stream = new NetworkStream(client))     using (StreamReader reader = new StreamReader(stream))     using (StreamWriter writer = new StreamWriter(stream))     {         // The client will first ask for a file, terminated by a newline;         // we respond by reading the file, sending it, and closing the stream.         string fileName = reader.ReadLine();         writer.Write(File.ReadAllText(fileName));     } } 

Client-Side Sockets

A client simply communicates with existing end points on the network. Therefore, there is no need to bind, listen, or accept. In fact, the only operation necessary is connect, aside from sending and receiving data, of course.

Connecting to an End Point

To connect to an existing end point, you must simply create a new Socket and then make a call to the Connect method. As usual, an asynchronous BeginConnect/EndConnect pair is also available. If something should go wrong in the process of connecting to the target end point, a SocketException will be generated. This has a few overloads that take such things as IPAddress and port numbers, string-based DNS host names, and the like. Note that we discuss how to generate end-point references, in particular through DNS, later in this section.

When calling Connect, the OS does have to allocate a network end point through which communication will be sent. While this is not strictly necessary, you can manually execute a Bind to specify which port to use. The following example shows the standard block of code used to connect to an end point:

 using (Socket s = /*...*/) {     s.Connect("wrox.com", 80);     // Read from and write to the socket... } 

Once you've made a successful connection, it is business as usual. You'll communicate using the Send and Receive commands described above.

Network Information

In situations where you need to obtain general information about the state of the network around you, a number of APIs can come in useful. These enable you to query things such as your network interface card (NIC) status, current network traffic, Domain Name System (DNS) and Internet Protocol (IP) address information, and even ping status of remote servers. We won't discuss all of them in detail, but they are available in the System.Net.NetworkInformation namespace and are mostly self-explanatory.

A number of constructors exist for the IPEndPoint class, allowing you to specify both an IP address and port for the socket to bind to. In addition to the IPAddress.Loopback you saw above, the System.Net .IPAddress type provides a number of ways with which to construct new IP references, including a static Parse method, which turns a string such as "111.22.3.44" into an IPAddress instance.

The System.Net.Dns class is also useful for discovering IP end points through DNS host names. It provides a set of static methods through which IPHostEntry instances can be constructed. Host entries contain such information as the string-based HostName, along with IP addresses for a target host. For example, this code obtains a new IPEndPoint by resolving the DNS name "wrox.com":

 IPEndPoint ep = Dns.GetHostByName("wrox.com").AddressList[0]; 

The NetworkInterface type offers methods to query the current network activity status. For example, the GetIsNetworkAvailable method returns a bool indicating whether the network is available or not. The Ping type permits you to issue standard pings using the Send or SendAsync methods, which can be used to detect connectivity to a specific destination.

Protocol Clients and Listeners

Protocols are complex beasts. While it is certainly possible to deal with raw byte exchange at the socket level, there are a number of APIs that raise the level of abstraction. Specifically, the various client and listener types in System.Net abstract away all of the low-level details of protocol exchange. The supported protocols and corresponding client types are: TCP/IP (TcpClient, TcpListener), UDP (UdpClient), HTTP and FTP (WebClient, HttpListener), and SMTP (SmtpClient). We'll take a look at each of these in this section.

The Open Systems Interconnection (OSI) model describes a network layer cake that is comprised of seven interesting layers. The purpose of each layer is to take responsibility for a specific level of abstraction on top of which other layers and systems may be built. Figure 7-1 depicts this model.

Figure 7-1: The Open Systems Interconnection (OSI) stack.

TCP/IP

The Transmission Communication Protocol (TCP) and Internet Protocol (IP) partnership is the killer combo that fuels most of the Internet. While a complete discussion is well beyond the scope of this book — indeed entire books are available (see "Further Reading") that describe nothing but communication protocols — it helps to briefly touch on what these protocols are useful for. We then see how the TcpClient and TcpListener classes enable you to write client and server-side applications that communicate using these transport protocols.

IP sits at layer 3 in the OSI stack. Its purpose is to provide a software interface to the underlying data link, and it specifies the way in which packets are transmitted and routed. It makes no guarantee as to the reliability of communication. IP also specifies the means of routing between addresses, two primary versions of which are in use today: IPv4 uses a 32-bit addressing scheme, while IPv6 utilizes 128-bits. IPv6 is generally considered the next generation IP addressing scheme, although it has not yet obtained widespread adoption. The .NET Framework supports both, although it is entirely dependent on the OS on which your application runs to provide the low-level support. IPv6 support on Windows XP, for example, was introduced with Service Pack 1, while Windows Server 2003 fully supports it. Windows 2000 and 98 do not support IPv6 at all.

TCP is a layer on top of IP. It sits at layer 4 in the OSI stack and provides a connection-based reliable means of transmitting network packets. It uses handshakes, checksums, and acknowledgments to establish reliable connections between end points. This enables TCP to determine whether packets were received and, if not, to retransmit them as necessary. It furthermore guarantees that packets are consumed in the correct order, something that IP by itself does not ensure.

Client-Side TCP/IP

The client type for communicating over TCP/IP, TcpClient, is really just a thin wrapper on top of a client Socket configured with the proper TCP/IP settings. Nonetheless, it can be more convenient than constructing and managing a raw socket itself. The Connect method overloads are identical to Socket's, and reading and writing is accomplished through the NetworkStream accessible from the GetStream method. The behavior of this type is otherwise the same as when working with sockets.

Listening for TCP/IP Connections

TcpListener is similarly just a thin wrapper on top of a server-side Socket. Its constructors take an argument representing which local port to listen on, and respond by configuring an underlying socket for TCP/IP communication. You must then execute Start, which causes an underlying Bind and Listen (if a backlog is specified with an argument to Start). The idioms from this point on are identical to those encountered when working with sockets, for example calling either AcceptSocket or AcceptTcpClient, which block until a client request comes in and provides access to the request via a Socket or TcpClient instance. When it's done, executing a Stop will destroy the underlying socket.

UDP

The User Datagram Protocol (UDP) is a protocol that, like TCP, builds on top of IP and sits at layer 4 in the OSI stack. UDP does not guarantee delivery and ordering of packets, nor does it prevent duplication of data. It does use checksums to prevent corruption of packets.

UDP is an appropriate solution for applications that stream real-time data that can accept data loss, for example, and that cannot incur the expense and overhead involved in using TCP. Generally, such applications need to send dense quantities of data over a long period of time. A great example of this category of system is media-streaming technologies. UDP builds on top of IP, and therefore the same IPv4 and IPv6 addressing comments noted above in the context of TCP/IP apply.

Like the TCP classes discussed above, UdpClient is a thin wrapper on top of an underlying socket that is properly configured for UDP/IP communication. Perhaps the only interesting difference between TcpClient and UdpClient worth noting is UDP's support for multicast messaging. Typically, a single client end point communicates with just one other server end point. In some situations, it is useful for a single program to broadcast a message to many end points, each of which has subscribed to be delivered such notifications. This is called multicasting. With UdpClient, you may both receive and send multicast messages.

In order to receive multicast messages, you need to join a multicast group. This is done with the Join MulticastGroup method, which must be providing the target group to join (in the form of an IPAddress). You can also specify the timeToLive, which is the maximum number of router hops a packet can travel before it will be discarded by the UdpClient. The DropMulticastGroup method unsubscribes your end point from the joined group. Sending a multicast message is as simple as delivering a message to an end point that has been set up as a multicast group. For example, this code snippet joins a multicast group, processes incoming broadcasts for a while, and then leaves the group:

 using (UdpClient udp = new UdpClient(1024)) {     IPAddress groupAddress = IPAddress.Parse("0.0.0.0");     udp.JoinMulticastGroup(groupAddress, 32);     udp.EnableBroadcast = true;     bool keepProcessing = true;     while (keepProcessing)     {         IPEndPoint sentBy = null;         byte[] data = udp.Receive(ref sentBy);         // Process data...         keepProcessing = /*...*/;     }     udp.DropMulticastGroup(groupAddress); } 

Notice that the Receive method uses a slightly unconventional format. It returns the bytes read from the incoming message, and sets a reference parameter to the IPEndPoint that was responsible for sending the message.

HTTP

The HyperText Transfer Protocol (HTTP) is the glue that binds the World Wide Web (WWW) together. It adds on top of TCP/IP support for content addressing mechanisms, called Uniform Resource Locators (URLs), and sits at OSI layer 7. HTTP communicates over port 80 by default, although this is configurable using your web server software and/or network library. HTTPS enables a layer of security through encryption and public/private key infrastructure. This is simply HTTP over Secure Socket Layer (SSL) — which sits at OSI level 5 — over TCP/IP.

Clients initiate actions using requests sent to a URL, and servers listening at that address recognize and respond to a set of standard HTTP actions (e.g., GET, POST, PUT). For example, the client request "GET/somepage.html" asks the server to obtain and send the page somepage.html. A server (e.g., a web server) will respond by sending a response code and additional data such as the page requested, headers, cookies, and so forth, in addition to the raw content.

ASP.NET provides a first class way to plug in to the Internet Information Services (IIS) web request and response pipeline — the web server that ships with Windows — and enables you to create dynamic web applications without needing to know very much about the HTTP protocol at all. But in addition to that, the System.Net APIs provide lightweight and powerful means of performing HTTP client and server programming.

Client-Side HTTP

The WebClient class provides a set of methods to generate client-to-server web requests. The method OpenRead takes either a string or System.Uri, specifying a location to which to open a connection and download data. It returns a Stream, meaning that it blocks for as long as it takes to obtain an initial response from the server and also provides a more efficient means for walking through data a few bytes at a time. The DownloadData and DownloadString methods are similar, returning a byte[] or string, respectively. These methods download the entire contents before unblocking and returning a result. If you intend to use only pieces of the data at a time, this can be an inefficient approach when compared to reading piece by piece using a Stream. DownloadFile takes an additional fileName argument representing a file in which to store the downloaded contents directly. The GetWebRequest method returns an instance of the WebRequest class, giving you more flexibility in terms of how requests and responses are carried out.

For example, this snippet shows some examples of each of these methods:

 WebClient client = new WebClient(); // The whole contents of this file will be processed at once: byte[] binaryBlob = client.DownloadData("file://server/myapp/objects.ser"); object[] blobContents = DeserializeBlob(binaryBlob); // Read in the whole document so we can load it into a DOM: XmlDocument doc = new XmlDocument(); doc.LoadXml(client.DownloadString("http://www.wrox.com/xmldocument.xml")); // Walk through the contents of this GZIP file piece by piece: using (StreamReader reader = new StreamReader(     new GZipStream(client.OpenRead("http://www.wrox.com/data.tgz"),     CompressionMode.Decompress))) {     // Use normal StreamReader functionality... } // Download the same GZIP file above and save it to disk: client.DownloadFile("http://www.wrox.com/samples.tgz",@" c:\samples.tgz"); 

Each of these methods offers an asynchronous version (suffixed by Async) and follows a slightly different pattern than shown elsewhere in this book. The method does not return any value and instead will make a callback to an event on the WebClient object. To process the return value, you must subscribe to the event that corresponds to the begin operation you perform (e.g., DownloadDataCompleted for DownloadDataAsync, and so forth). Calling the CancelAsync method cancels pending asynchronous workers.

In addition to the asynchronous completion events, you can use the DownloadProgressChanged event in order to monitor ongoing download progress, whether synchronous or asynchronous. This is useful when you need to keep a user updated regarding downloading progress, for example, and is triggered upon each completed round trip.

You have manual control over the credentials sent for authentication through the Credentials property, or you can have the default credentials the process is running under sent by setting UseDefault Credentials to true. If you need to tunnel through a proxy server, you can set this through the Proxy property.

Listening for HTTP Connections

Available as of Windows XP Service Pack 2 (SP2) and Windows Server 2003 is a simple kernel-mode HTTP listener. This enables you to plug into the Windows HTTP pipeline without having to explicitly use IIS, at the same time providing much of the same features and functionality, albeit with a sacrifice in configuration and administration capabilities. This new server is called HTTP.SYS. The HttpListener class surfaces much of the ability of this new listener. It even provides authentication and authorization configuration, similar to what you can get by choosing IIS.

Start to use the HttpListener is simple: just create a new instance, set the appropriate end points to listen against through the Prefixes property (i.e., HTTP or HTTPS end point URLs to which the listener will bind), and call the Start method to begin accepting incoming connections. By default, the listener will run on port 80 for HTTP and port 443 for HTTPS and allows anonymous/unauthenticated clients. You can specify a custom port with the Prefixes URLs and can configure the authentication settings by setting the AuthenticationSchemes property. AuthenticationSchemes is a flags-style enumeration, meaning that you can set any combination of authentication types instead of the default of only Anonymous:

• Anonymous: The default value. This allows any client to connect without providing any authentication information.

• Basic: Challenges client connections to provide a clear-text (base-64 encoded) username and password. This is not an overly secure approach because credentials can easily be sniffed and interpreted off the network. For simple authentication, this might be sufficient.

• Digest: Similar to basic authentication, but username and passwords are transmitted using a simple cryptographic hash.

• Ntlm: This will prompt the user for NT LAN Manager (NTLM) credentials. Internet Explorer supports this type of authentication and can easily be configured to automatically send a user's NTLM token based on browser settings. Other browsers do not support NTLM at all.

• IntegratedWindowsAuthentication: Requires that the client support the strongest level of authentication enabled on the server. For example, if the server uses Kerberos, the client must provide a Kerberos ticket. Otherwise, authentication utilizes NTLM.

• Negotiate: This will negotiate between the client and server to find and use the strongest authentication mechanism based on common levels of support. If both support Kerberos, this is preferred; otherwise NTLM is used.

Regardless of which scheme you use, authentication is performed against user repositories that the server is setup to use. In many cases this means using a domain server, local user authentication, or Active Directory (AD) or other authentication repositories that support Lightweight Directory Access Protocol (LDAP). Details about administering such services is not within the scope of this book.

Processing Incoming HTTP Requests

Once you have set up your HttpListener, you are ready to start processing incoming requests. GetContext is similar to socket's Accept in that it blocks until a request is ready to process, and returns an object (of type HttpListenerContext) through which you may access the client. Similarly, BeginGet Context and EndGetContext are the asynchronous versions of this API.

A client request is represented by an instance of HttpListenerContext. The Request property on this object offers a great deal of information regarding the client's request, represented by a HttpListener Request instance. Through this object can be found the various HTTP headers sent with the user's request, the requested URL and query-string, any client cookies, and so on. User retrieves the authenticated principal (if any) of type System.Security.Principal.IPrincipal, which can be used for, among other things, authorization using the IsInRole method. This type is described further in Chapter 9.

Response returns a HttpListenerResponse object that can be used for responding to the client request. This enables you to set the response code, send a client-side Redirect to a separate URL, send cookies, set headers, and of course write data to the client using the OutputStream property. Once you're done with a particular request, calling Close will end the connection.

The simple example in Listing 7-5 demonstrates an echo server, which just listens for and prints back out the name parameter in the query-string. A real application would probably want to use similar techniques to those shown above with server-side sockets to process incoming requests asynchronously on the ThreadPool.

Listing 7-5: A simple HTTP echo server

 using (HttpListener listener = new HttpListener()) {     // Set up our listener:     listener.AuthenticationSchemes = AuthenticationSchemes.Negotiate;     listener.Prefixes.Add("http://localhost:8080/echo/");     listener.Prefixes.Add("https://localhost/echo/");     listener.Start();     // Keep processing requests until we shut-down (i.e. process exit):     // (We could probably make this more user-friendly.)     while (true)     {         // Wait for the next incoming connection:         HttpListenerContext ctx = listener.GetContext();         ctx.Response.StatusCode = 200; // HTTP OK         string name = ctx.Request.QueryString["name"];         // Now use a writer to respond:         using (StreamWriter writer = new StreamWriter(ctx.Response.OutputStream))         {             writer.WriteLine("<p>Hello, {0}</p>", name);             writer.WriteLine("<ul>");             foreach (string header in ctx.Request.Headers.Keys)             {                 writer.WriteLine("<li><b>{0}:</b> {1}</li>",                     header, ctx.Request.Headers[header]);             }             writer.WriteLine("</ul>");         }         ctx.Response.Close();     }     listener.Stop(); } 

E-Mail (SMTP)

The Simple Mail Transport Protocol (SMTP), like HTTP, sits at layer 7 in the OSI stack. It enables programs to send simple text e-mails. SMTP works by exchanging a brief conversation between client and server, which includes the client specifying the target e-mail address, subject line, and message content, among other things. By default, SMTP communication occurs over port 25. Most e-mail messages are formatted using Multipurpose Internet Mail Extensions (MIME), providing the capability to send data other than just 8-bit ASCII-encoded text messages. This can be used, for example, to send Unicode messages and to include binary attachments.

Sending E-Mail

The SmtpClient class is to SMTP what the WebClient class is to HTTP, providing a lightweight means to send SMTP e-mail messages. To use it, first construct an instance, optionally providing the SMTP server and port through which messages will be routed. The Host and Port properties enable you to change these once an instance is already created. By default, it will use the mail configuration settings on the current machine. Most enterprise servers are set up to route SMTP through a central outbound SMTP server. If the target SMTP server requires authentication, you will want to either manually set Credentials or change UseDefaultCredentials to true.

The simplest way to send a message is then to invoke the void Send(string from, string recipients, string subject, string body) overload. This will generate an e-mail that is from the e-mail specified by from, to the provided recipients, with the specified subject and body. If you want more control over the message format, need to add attachments, or want to specify CC (copy) or BCC (blind copy) recipients, you will need to use the void SendMail(MailMessage message) overload. Both Send and SendMail offer asynchronous versions that follow the same pattern described above for the WebClient class.

MailMessage provides several interesting constructor overloads and properties. It has From, To, Subject, Body, CC, and Bcc properties with which to specify the primary pieces of e-mail information. The recipient properties are actually collections, meaning that you are able to specify multiple addresses for each. Addresses are represented by instances of the MailAddress type and permit you to specify an optional display name in addition to the raw e-mail address. MailMessage also has a property Attachments, which is a collection of Attachment instances. Setting up an instance is really just a matter of calling the right constructor, passing in either a filename or Stream from which to read the attachment data.

When constructing an Attachment you must provide a name, either via a constructor parameter or with the Name property. You should also consider specifying the MIME type (a.k.a. mediaType) so that clients are able to recognize the file type correctly. This ensures mail clients are able to launch the correct associated program to read the file. The easiest way to discover a file's MIME type is to examine your own computer's file associations or by searching the Internet.

 SmtpClient client = new SmtpClient("smtp.localhost.com"); // If your outgoing SMTP server requires logon information, // the following line ensures it will be sent: client.Credentials =     new NetworkCredential("username", "password"); // Construct our message, set the from, to, CC, subject, and body: MailMessage message = new MailMessage(     new MailAddress("[e-mail]", "Joe Duffy"),    // To     new MailAddress("[e-mail]", "Raj Duffy")); // CC message.Bcc.Add(new MailAddress("[e-mail]")); message.Bcc.Add(new MailAddress("[e-mail]")); message.Subject = "..."; message.Body = "..."; // Attach a file from disk: Attachment att = new Attachment(@" c:\path\to\file.cs"); message.Attachments.Add(att); // Perform the send operation: client.Send(message); 

This sends a message to a single recipient ([e-mail]), with a single CC recipient ([e-mail]) and two BCC recipients ([e-mail] and mika@ferret spaceships.com). Included is a single file that gets read from the local path C:\path\to\file.cs.

Legacy Mail APIs

Prior to version 2.0 of the .NET Framework, it was possible to send e-mail over SMTP through the System.Web.Mail libraries. The core type used from this namespace is SmtpMail, offering a set of static members. The SmtpServer property is used to specify the server to route messages through (for all messages sent inside the AppDomain), and a set of Send methods are used to queue up outbound mail messages. Unfortunately, these libraries use the Collaboration Data Objects (CDO) COM APIs that ship with Windows 2000 and up (versions for earlier OSes are available with Exchange Server and Outlook). These APIs need to interoperate with COM and are generally significantly less efficient than the new APIs introduced with the System.Net.Mail namespace.