4.6 Threading in the Pipeline

To efficiently service multiple client requests , Web servers make extensive use of concurrency, by launching multiple processes and/or spawning multiple threads to service requests and perform work. As we have seen, ASP.NET creates a distinct AppDomain for each application housed in its worker process, but what has not yet been mentioned is how threads are allocated and dispatched to service those requests.

For the most part, ASP.NET developers need not concern themselves with the multitude of issues that come with developing in a multithreaded environment. Page requests are always serviced on the same thread, and a distinct instance of the Page class is always created to service any new requests. Distinct instances of application and module objects are also used to service each request. It is important, however, to understand how threads are used to service requests, so that you don't make any incorrect assumptions about concurrent access to any of the state in your applications.

To begin with, ASP.NET uses the process-wide CLR thread pool to service requests. The size of this pool is configurable in the processModel element of machine.config , discussed in Chapter 3, and is set to a default of 25 worker threads and 25 I/O threads. When running on Windows 2000 or Windows XP, ASP.NET services requests primarily on I/O threads from the CLR thread pool. This is done for efficiency because each request is initiated by an asynchronous write to a named pipe from the ISAPI extension DLL, aspnet_isapi.dll, within the IIS process ( inetinfo.exe ). When the asynchronous write is received by the ASP.NET worker process ( aspnet_wp.exe ), it is processed on an I/O thread, so to avoid thread switching, the request is usually serviced directly on that thread. Note that this behavior changes with Windows Server 2003 and IIS 6.0 because ASP.NET is more tightly integrated. In IIS 6.0, there is no dedicated worker process for ASP.NET since it is integrated into the process model exposed by IIS 6.0, which lets you designate whether a particular virtual directory lives in a distinct worker process ( w3wp.exe ) or in a worker process shared by other virtual directories. In this scenario, ASP.NET services requests on worker threads drawn from the process-wide CLR thread pool.

For each request that comes in, a new instance of the appropriate HttpApplication -derived class is created, as are the associated modules for that application. To avoid reallocating applications and modules too often, each AppDomain maintains a pool of applications and modules. The maximum size of the application pool is the same as the size of the thread pool, so by default, up to 25 requests per worker process can be processed concurrently, each with its own application and module set. Figure 4-5 shows a possible snapshot in time of the ASP.NET worker process. In this scenario, there are two active applications in the worker process, each with a dedicated AppDomain . Each application is currently processing two requests, and each is using two threads from the ASP.NET thread pool to service those requests.

Figure 4-5. Threading and Pooling in the HTTP Pipeline

Several aspects of this architecture may influence the way you construct your ASP.NET applications. First of all, the fact that applications and modules are instantiated multiple times for a particular application means that you should never rely on adding fields or other state to your application or module classes, because it is replicated and not shared across multiple requests as you might think. Instead, use one of the many state repositories available in the pipeline, such as the application-wide cache, the session state bag, the application state bag, or the per-request items collection of the HttpContext class. Also, by default, most handlers created to service requests are not pooled. As we have seen, you can pool handlers and even control pooling on a per-handler basis via the IsReusable property of IHttpHandler , but the only handlers that are pooled implicitly are custom handlers that you write with no designated handler factory. The PageHandlerFactory class does not perform pooling, nor does the factory SimpleHandlerFactory class, which instantiates .ashx-defined handlers. Typically, therefore, each request is serviced by a freshly allocated instance of the appropriate handler class, which is discarded after the request completes.

4.6.1 Asynchronous Handlers

When we discussed implementing custom handlers earlier in the chapter, the call to the ProcessRequest method of the handler was always made synchronously in the context of the thread executing the request in the pipeline. There may be occasions where you would like this invocation to be asynchronous, allowing the primary request processing thread to be returned to the thread pool while your handler performs its work. For these occasions, there is another handler interface, derived from IHttpHandler , called IHttpAsyncHandler , shown in Listing 4-23.

Listing 4-23 The IHttpAsyncHandler Interface

 Public Interface IHttpAsyncHandler        Inherits IHttpHandler   Function BeginProcessRequest(ctx As HttpContext, _                                cb As AsyncCallback, _                                obj As Object) As IAsyncResult   Sub EndProcessRequest(ar As IAsyncResult) End Interface 

Handlers that implement this interface must implement two additional methods beyond the standard methods of IHttpHandler . The first method is BeginProcessRequest , which the application class calls instead of directly calling ProcessRequest . It is then up to the handler to launch a new thread to process the request, and return immediately from the BeginProcessRequest method, passing back a reference to a class that implements IAsyncResult so that the runtime can detect when the operation is complete. The other method, EndProcessRequest , is called when the request processing is complete, and can be used to clean up any allocated resources if necessary.

The most straightforward ways to implement an asynchronous handler would be either to use an asynchronous delegate invocation or to call ThreadPool.QueueUserWorkItem with the method to perform the request processing. Unfortunately , using either of these two techniques would completely defeat the purpose of building an asynchronous handler, because they both draw from the same process-wide CLR thread pool that ASP.NET uses to service requests. While the primary request thread would indeed be freed up and returned to the thread pool, another thread would be drawn out of the pool to perform the asynchronous delegate execution (or work item completion), resulting in a net gain of zero threads for servicing additional requests and thus rendering the asynchronous nature of the handler useless.

To build a truly effective asynchronous handler, therefore, you must spawn an additional thread by hand in response to BeginProcessRequest (or even better, use a thread from a different thread pool, discussed later). There are three important aspects to building a successful asynchronous handler:

• Constructing a class that supports IAsyncResult to be returned from BeginProcessRequest

• Spawning the thread to perform your request processing asynchronously

• Notifying ASP.NET that you are finished processing the request and are ready to return the response

We begin the construction of an asynchronous handler by building a class that supports IAsyncResult . This class will be returned from the call to BeginProcessRequest and later will be passed into our implementation of EndProcessRequest , so among other things, this class is a useful place to store request-specific state that we may need to use during the processing of a request. The IAsyncResult interface is shown in Listing 4-24.

Listing 4-24 The IAsyncResult Interface

 Public Interface IAsyncResult   Public ReadOnly Property AsyncState As Object   Public ReadOnly Property CompletedSynchronously _                            As Boolean   Public ReadOnly Property IsCompleted As Boolean   Public ReadOnly Property AsyncWaitHandle As WaitHandle End Interface 

In our example, we store a reference to the HttpContext object associated with this request, a reference to the AsyncCallback delegate passed into BeginProcessRequest (which we must later invoke to complete the request), and a generic object reference for extra data that may be used by the caller of BeginProcessRequest . The other element that must be implemented in this class is a synchronization object that threads can wait on to be signaled when the operation completes. We use the common technique of supplying a ManualResetEvent that fires when our request is complete, but we allocate it only if someone requests it. Finally, our class has a convenience method called CompleteRequest that triggers the ManualResetEvent if it was created, invokes the AsyncCallback delegate, and sets our IsCompleted flag to true . The complete class definition for AsyncRequestState is shown in Listing 4-25.

Listing 4-25 The AsyncRequestState Class Definition

 Class AsyncRequestState       Implements IAsyncResult   Public Sub New(ctx As HttpContext, cb As AsyncCallback, _                  extraData As object )     _ctx = ctx     _cb = cb     _extraData = extraData   End Sub   Friend _ctx As HttpContext   Friend _cb As AsyncCallback   Friend _extraData As Object   private _isCompleted  As Boolean = false   private _callCompleteEvent As ManualResetEvent   Friend Sub CompleteRequest()     _isCompleted = true     SyncLock Me       If Not _callCompleteEvent Is Nothing Then         _callCompleteEvent.Set()       End If     End SyncLock     ' if a callback was registered, invoke it now     If Not _cb Is Nothing Then       _cb(Me)     End If   End Sub   ' IAsyncResult   '   Public ReadOnly Property AsyncState As Object _          Implements IAsyncResult.AsyncState     Get       Return _extraData     End Get   End Property   Public ReadOnly Property CompletedSynchronously _            As Boolean _            Implements IAsyncResult.CompletedSynchronously     Get       Return False     End Get   End Property   Public ReadOnly Property IsCompleted As Boolean _          Implements IAsyncResult.IsCompleted     Get       Return _isCompleted     End Get   End Property   Public ReadOnly Property AsyncWaitHandle As WaitHandle _          Implements IAsyncResult.AsyncWaitHandle     Get       SyncLock Me         If _callCompleteEvent Is Nothing Then           _callCompleteEvent = New ManualResetEvent(false)         End If         Return _callCompleteEvent       End SyncLock     End Get   End Property End Class 

The next step is to spawn a new thread on which we will process our request. The method we call on this new thread will need access to the state we cached in the AsyncRequestState class shown in Listing 4-25, but unfortunately the ThreadStart delegate used to spawn new threads in .NET does not take any parameters. To get around this, we create another class with the necessary state cached as data members (in this case, simply a reference to the AsyncRequestState object for this request) and with an instance method that can be used to initialize the ThreadStart delegate. Listing 4-26 shows the definition of this class, called AsyncRequest . Note that the ProcessRequest method we define in this class is the method that will be called from our manually created thread, and when it completes, it signals that the request processing is complete by invoking CompleteRequest on the AsyncRequestState object.

Listing 4-26 The AsyncRequest Class Definition

 Public Class AsyncHandler        Implements IHttpAsyncHandler   Public Sub ProcessRequest(ctx As HttpContext) _          Implements IHttpAsyncHandler.ProcessRequest     ' not used   End Sub   Public ReadOnly Property IsReusable As Boolean _          Implements IHttpAsyncHandler.IsReusable     Get       Return False     End Get   End Property   Public Function BeginProcessRequest(ctx As HttpContext, _                        cb As AsyncCallback, _                        obj As object) As  IAsyncResult _          Implements IHttpAsyncHandler.BeginProcessRequest     Dim reqState As AsyncRequestState     reqState = New AsyncRequestState(ctx, cb, obj)     Dim ar As AsyncRequest     ar = New AsyncRequest(reqState)     Dim ts As ThreadStart     ts = New ThreadStart(AddressOf ar.ProcessRequest)     Dim t As Thread     t = New Thread(ts)     t.Start()     Return reqState   End Function   Public Sub EndProcessRequest(ar As IAsyncResult ) _          Implements IHttpAsyncHandler.EndProcessRequest     ' This will be called on our manually created thread     ' in response to our calling ASP.NET's AsyncCallback     ' delegate once our request has completed processing.     ' The incoming     ' IAsyncResult parameter will be a reference to the     ' AsyncRequestState class we built, so we can access     ' the Context through that class if we like.     ' Note - you *cannot* access the current context     ' using the HttpContext.Current property, because we     ' are running on our own thread, which has not been     ' initialized with a context reference.     If TypeOf ar Is AsyncRequestState Then         Dim ars As AsyncRequestState         ars = CType(ar, AsyncRequestState)       ' here you could perform some cleanup, write       ' something else to the Response, or do something else     End If   End Sub End Class 

Finally, we are ready to build the asynchronous handler class itself. This class, which we just call AsyncHandler , must implement all the methods of the IHttpAsyncHandler interface shown earlier, which derives from IHttpHandler , for a total of four methods. The ProcessRequest method is not used and will never be called, because we implement BeginProcessRequest . In BeginProcessRequest , we create a new instance of our AsyncRequestState class, initializing it with the HttpContext object, the AsyncCallback delegate, and the generic object reference passed in as parameters. We then prepare a new AsyncRequest object, initialized with the freshly created AsyncRequestState object, and launch a new thread. Our implementation of EndProcessRequest does not do anything in this example, but it can be used in general to perform any cleanup or last-minute response additions. The AsyncHandler class definition is shown in Listing 4-27.

Listing 4-27 The AsyncHandler Class Definition

 Public Class AsyncHandler        Implements IHttpAsyncHandler   Public Sub ProcessRequest(ctx As HttpContext) _          Implements IHttpAsyncHandler.ProcessRequest     ' not used   End Sub   Public ReadOnly Property IsReusable As Boolean _          Implements IHttpAsyncHandler.IsReusable     Get       Return False     End Get   End Property   Public Function BeginProcessRequest(ctx As HttpContext, _                        cb As AsyncCallback, _                        obj As object) As  IAsyncResult _          Implements IHttpAsyncHandler.BeginProcessRequest     Dim reqState As AsyncRequestState     reqState = New AsyncRequestState(ctx, cb, obj)     Dim ar As AsyncRequest     ar = New AsyncRequest(reqState)     Dim ts As ThreadStart     ts = New ThreadStart(AddressOf ar.ProcessRequest)     Dim t As Thread     t = New Thread(ts)     t.Start()     Return reqState   End Function   Public Sub EndProcessRequest(ar As IAsyncResult ) _          Implements IHttpAsyncHandler.EndProcessRequest     ' This will be called on our manually created thread     ' in response to our calling ASP.NET's AsyncCallback     ' delegate once our request has completed processing.     ' The incoming     ' IAsyncResult parameter will be a reference to the     ' AsyncRequestState class we built, so we can access     ' the Context through that class if we like.     ' Note - you *cannot* access the current context     ' using the HttpContext.Current property, because we     ' are running on our own thread, which has not been     ' initialized with a context reference.     If TypeOf ar Is AsyncRequestState Then         Dim ars As AsyncRequestState         ars = CType(ar, AsyncRequestState)       ' here you could perform some cleanup, write       ' something else to the Response, or do something else     End If   End Sub End Class 

If we now build this handler and register it as an endpoint (as we did with our earlier handlers), it will successfully process requests asynchronously from the calling request thread of ASP.NET. Figure 4-6 shows the sequence of events that take place when a request mapped onto our asynchronous handler is made. First, the application class notices that our handler implements IHttpAsyncHandler , so instead of calling the synchronous ProcessRequest method, it invokes BeginProcessRequest on our handler, passing in the current context and an asynchronous callback delegate for our handler to invoke when it is complete. Our handler then creates a new AsyncRequestState object, initialized with the parameters passed into BeginProcessRequest . Next, our handler creates a new AsyncRequest object, initialized with the AsyncRequestState object, and launches a new thread using the AsyncRequest.ProcessRequest method as the entry point. Our handler then returns the AsyncRequestState object to the application object, and the calling thread is returned to the thread pool while our hand-created thread continues processing the request.

Figure 4-6. Asynchronous Handler Operation ”Phase 1: The Handoff

Once the ProcessRequest method of our AsyncRequest object has finished performing its lengthy tasks , it calls the CompleteRequest method of our AsyncRequestState class. This in turn fires the AsyncCallback delegate originally passed into our BeginProcessRequest method, signaling that the response has been prepared and is ready for return. The first thing the AsyncCallback delegate does is to call the EndProcessRequest method on our asynchronous handler class. Once that returns, it triggers the completion of the request by sending back the prepared response. Note that all of this processing happens on the secondary thread that we created in our handler, not on a thread pool thread. Figure 4-7 shows the steps for completing the request to our asynchronous handler.

Figure 4-7. Asynchronous Handler Operation ”Phase 2: Request Completion

One problem remains with our asynchronous handler implementation ”it has the potential to create an unbounded number of threads. If many requests are made to our asynchronous handler, all of which take a significant amount of time to service, we could easily end up creating more threads than the underlying operating system could handle. To deal with this, we need to provide a secondary thread pool to service our asynchronous requests in a bounded fashion. The mechanics of creating custom thread pools are beyond the scope of this chapter, but a fully operational asynchronous handler using a supplemental thread pool is available for download in the online samples for this book.

Building asynchronous handlers instead of synchronous ones can add considerable complexity to a design, so you should take care when determining whether you really need the asynchronous capability of these types of handlers. The purpose of an asynchronous handler is to free up an ASP.NET thread pool thread to service additional requests while the handler is processing the original request. This makes sense to do only if the work of servicing the request requires a significant amount of non-CPU-bound time to complete. For example, if the completion of a request depended on the completion of several remote procedure calls or perhaps Web service invocations, that would be a candidate to implement an asynchronous handler. Building asynchronous handlers to service CPU- intensive requests only adds threads to compete with the ASP.NET thread pool threads and may slow down the overall processing time for the request.