Resource Management
When performing network operations, it is important to post multiple asynchronous receive operations to ensure that the application receives data as fast as possible. However, this practice can cause problems when the number of concurrent connections handled by a server increases . Additionally, as a server accepts more and more connections and performs send operations on each connection, the bandwidth of the local network also must be taken into consideration. For this reason, memory and bandwidth issues are important considerations when designing a scalable server. The following two sections will discuss these concepts in more detail.
Memory
Using the asynchronous I/O pattern is paramount for high performance, but rules still need to be followed to achieve scalability. For example, posting dozens rather than a few asynchronous receive operations on a Socket or Stream will not drastically increase performance, but it will increase the amount of memory used. The application can run out of resources, which will affect the number of connections an application can handle.
Consider a TCP Socket “ based server application that maintains multiple connections and posts varying numbers of asynchronous receive operations. Each BeginReceive posted requires a buffer to receive the data, plus a small structure to maintain context information for the operation (including a reference to the receive buffer). If each receive operation uses a 16-KB receive buffer, plus a 200-byte context structure, then each BeginReceive uses 16,584 bytes. Table 14-1 calculates the memory requirements for various connection counts and operations per connection where each operation requires 16,584 bytes.
| Total Connections | Operations Per Connection | Total Memory (Bytes) |
|---|---|---|
| 10 | 10 | 1,658,400 |
| 1,000 | 10 | 165,840,000 |
| 50,000 | 10 | 8,292,000,000 |
| 10 | 2 | 331,680 |
| 1,000 | 2 | 33,168,000 |
| 50,000 | 2 | 1,658,400,000 |
Notice that the operations per connection field can be any combination of asynchronous send and receive operations ”the basic idea is that each connection has the given number of operations outstanding. Posting 10 asynchronous operations for each connection limits a typical server to handling around 24,000 connections maximum ”a typical server being defined as an architecture running a 32-bit operating system at the maximum memory configuration possible of 4 GB. If the application keeps the number of outstanding operations on each connection down to two, you ll notice a significant increase in the number of connections that can be handled.
 |
The number of network connections the Microsoft Windows NT operating systems (Windows NT 4, Windows 2000, Windows XP, and Windows Server 2003) can establish is limited based on the memory resources available. The operating system reserves a portion of the total memory in what is known as non-paged memory. Non-paged memory contains information and data structures that are never paged out of memory. Usually, the system reserves one-quarter of the total memory for the non-paged pool, with a limit of 256 MB on Windows 2000 and later and 128 MB on Windows NT 4. These limits are for 32-bit versions of the operating system.
Operating system constructs, such as file handles, process information, networking connections, and so on, are examples of information that must always be resident in physical memory. Each TCP connection consumes approximately 2 KB of the non-paged memory. Because of this, a system with 256 MB of the non-paged pool can establish roughly 100,000 connections. Remember that a portion of the non-paged pool is being used by other system components , so networking cannot consume the entire amount. Also, the data buffers used to send and receive data must be locked into the non-paged pool while the network stack processes data.
| |
Socket operations will start to fail if a server reaches a point where there is too little free memory. In this case, a SocketException is thrown where the ErrorCode property is the Winsock error code WSAENOBUFS (10055). The Socket should be closed when this occurs to free all associated resources and ensure other operations on different sockets don t also fail due to insufficient memory.
Lastly, a server should guard against idle connections, which are a form of attack. Consider a request-response “based server that accepts client connections, waits for a request, and issues a response. If a client connects but never sends the request, the server typically posts an asynchronous receive that never completes. If enough malicious clients do this, valid clients can be prevented from connecting as the server will run out of resources. A defensive server should keep track of how long each client is idle and close the connection if it exceeds a predefined limit.
Bandwidth
When performing network operations, the bandwidth of the network directly affects how well the application scales . For example, an FTP-like application that sends or receives large data in bulk is not going to scale well past a few hundred connections on a 10-MB network. When designing network-based applications, responsiveness is an important design goal. If the FTP server allowed 1,000 concurrent users, each connection would rate at 1,250 bytes/second. This isn t terrible unless the file being retrieved is several megabytes in size . Table 14-2 lists transfer rates for concurrent connections and local network bandwidths.
| Total Connections | Network Bandwidth (megabits) | Bytes/Second per Connection |
|---|---|---|
| 100 | 10 | 12,500 |
| 1,000 | 10 | 1,250 |
| 50,000 | 10 | 25 |
| 100 | 100 | 125,000 |
| 1,000 | 100 | 12,500 |
| 50,000 | 100 | 250 |
The local bandwidth plays an important role in establishing limits on the number of concurrent asynchronous operations to allow per connection. For example, if a server is designed to handle 1,000 concurrent connections on a 100-MB network, each connection can send and receive at 12,500 bytes/second. Posting 10 asynchronous sends of 8 KB each is a waste of resources as there always will be eight operations waiting on the network to send the data. An efficient design limits two asynchronous sends per connection.
If a network becomes overly congested , packets will be lost or dropped and the TCP protocol will be forced to retransmit packets. This causes more congestion, and it s likely that TCP will timeout when the recipient fails to acknowledge it received the data, which causes the network stack to abort the connection. When this connection is dropped, any pending socket operation or a subsequently issued socket method will fail with a SocketException . The ErrorCode value will be the Winsock error WSAECONNABORTED (10053). If a server experiences an excessive number of aborted connections, it should disallow additional connections until the number of currently established clients drops below a threshold. Another option is for it to close valid connections to lower network congestion. If no action is taken, it s probable that other accepted connections will be aborted leading to many failed clients ”many more than if the server preemptively closes a number of connections to bring the network congestion down so that the remaining clients can be successfully serviced.
If an application efficiently handles sending data, it also must efficiently handle receiving data. As we mentioned earlier, if an application does not receive data fast enough, the TCP window size shrinks, which throttles the sender from sending additional data until the receiver catches up. The rule of thumb for receiving is to ensure at least one asynchronous receive is posted at all times. To do this, an application should have three or four receive operations posted at any given time. This allows the application to process one operation while several are still outstanding and the network stack can fill those buffers as data arrives. Again, it is important that the receive callback does not take too long to process. Of course, this rule applies only to applications that receive data at a high rate. A single asynchronous receive is sufficient for applications that receive small chunks of data infrequently.
EAN: 2147483647
Pages: 121