Using Service Throttling to Control Resource Use | Microsoft Windows Communication Foundation Step by Step (Step By Step Developer Series)

You can use service throttling to prevent over-consumption of resources in a WCF service. If you recall from Chapter 10, “Programmatically Controlling the Configuration and Communications,” when a message received by a service host reaches the top of the channel stack, it passes to a ChannelDispatcher object, which in turn passes it to the appropriate EndpointDispatcher object, which invokes the corresponding method in the appropriate service instance. However, before forwarding the request to the EndpointDispatcher object, the ChannelDispatcher object can examine the current load on the service and elect to hold the request back if the request would cause the service to exceed the permissible load. The request is blocked and held in an internal queue until the load on the service eases. The ChannelDispatcher object has a property called ServiceThrottle that you can use to help control how the ChannelDispatcher decides whether to block and queue requests or let them execute. The ServiceThrottle property is an instance of the ServiceThrottle class, and this class itself exposes three further integer properties:

MaxConcurrentInstances. This property specifies the maximum number of concurrent service instances that the service host will permit.
MaxConcurrentCalls. This property specifies the maximum number of concurrent messages that the service host will process. If a client application makes a large number of concurrent calls, either as the result of invoking OneWay operations or by using client-side multi-threading, it can quickly monopolize a service. In this scenario, you might want to limit each client to a single thread in the service by setting the ConcurrencyMode property of the service to ConcurrencyMode.Single. The client application can continue running asynchronously and should remain responsive to the user, but requests submitted by the client application will be processed in a serial manner by the service.
MaxConcurrentSessions. This property specifies the maximum number of concurrent sessions that the service host will permit. Client applications are responsible for establishing and terminating sessions and can make several calls to the service during a session. Clients creating long-running sessions can result in other clients being blocked, so keep sessions as brief as possible and avoid performing tasks such as waiting for user input.

Configuring Service Throttling

By default, the ServiceThrottle property of the ChannelDispatcher object is null and the WCF runtime uses its own default values for the maximum number of concurrent instances, calls, and sessions. To control scalability, you should arrange for the WCF runtime to create a ServiceThrottle object and explicitly set these properties to values suitable for your environment, taking into account the expected number of concurrent client applications and the work that they are likely to perform. You can perform this task in code by creating a ServiceThrottlingBehavior object, setting its properties (the ServiceThrottlingBehavior class provides the same properties as the ServiceThrottle class), and adding it to the collection of behaviors attached to the ServiceHost object, as described in Chapter 10. You must do this before opening the ServiceHost object. The following code shows an example:

 // Required for the ServiceThrottlingBehavior class using System.ServiceModel.Description; … ServiceHost host = new ServiceHost(…); ServiceThrottlingBehavior throttleBehavior = new ServiceThrottlingBehavior(); throttleBehavior.MaxConcurrentCalls = 40; throttleBehavior.MaxConcurrentInstances = 20; throttleBehavior.MaxConcurrentSessions = 20; host.Description.Behaviors.Add(throttleBehavior); host.Open(); …

However, be warned that the values of the properties in a ServiceThrottle object can have a drastic affect on the response time and throughput of a WCF service. You should actively monitor the performance of the WCF service and be prepared to change these settings if the computer hosting the service is struggling. Additionally, clients blocked by limits that are set too low can result in an excessive number of time-outs or other errors occurring in the client application or the channel stack, so be prepared to catch and handle them.

Note

At the time of writing, the default value for the maximum number of concurrent instances is 21474836467 (Int32.MaxValue), the default value for the maximum number of concurrent calls is 16, and the default value for the maximum number of concurrent sessions is 10. However, these values may change in subsequent releases of WCF.

A more flexible way to create a ServiceThrottle object and set its properties is to add a service behavior that contains the <serviceThrottling> element to the service configuration file. This is the approach that you will adopt in the following exercise.

Apply Throttling to the ShoppingCartService Service

Using Visual Studio 2005, open the solution file ShoppingCartService.sln located in the Microsoft Press\WCF Step By Step\Chapter 12\Throttling folder under your \My Documents folder.

This solution contains a simplified non-transactional version of the ShoppingCartService service that does not actually update the database, and an extended version of the client application that opens multiple concurrent sessions to the service.

Note

The rationale behind not updating the database or using transactions is to allow you to concentrate on the throttling semantics of a service and not worry about any potential locking and concurrency issues in the database. In the real world, you have to take all of these factors into account.

Open the ShoppingCartService.cs file in the ShoppingCartService project. Notice that the service specifies that sessions are required in the ServiceContract attribute of the IShoppingCartService interface and that the ServiceBehavior attribute of the ShoppingCartServiceImpl class specifies the PerSession instance context mode.
Examine the AddItemToCart method in this class. This method starts with a WriteLine statement that displays the method name. A corresponding WriteLine statement has been added at each point that the method can terminate. You will use these statements to trace the progress of each instance of the service as it runs. Also notice that the method contains the statement System.Threading.Thread.Sleep(5000) immediately after the first WriteLine statement. Although this method still queries the database, it no longer performs updates for reasons described earlier. This statement slows the method down by waiting for 5 seconds, simulating the time taken to perform the database update (assume the database update operation is very time consuming), and making it a little easier to observe the effects of the service throttling parameters. The other public methods, RemoveItemFromCart, GetShoppingCart, and Checkout, have been amended in the same way.
Open the Program.cs file in the ShoppingCartClient project, and locate the doClientWork method. This method contains code that creates a new instance of the proxy object and then invokes the various operations in the ShoppingCartService service, in much the same way as you have seen in earlier chapters. The method contains WriteLine statements displaying its progress to the console. The output includes a number that identifies the client. The client connects to the service using a standard TCP binding.
Examine the Main method. This method asynchronously calls the doClientWork method ten times, passing in the client number as the parameter. Each call creates a new thread. This simulates ten different but identifiable clients connecting to the service at the same time.
Open the Program.cs file in the ShoppingCartHost project. This is the application that hosts the service. Add the following using statement to the top of the file:
```
 using System.ServiceModel.Dispatcher;
```
This namespace contains the ServiceThrottle and ChannelDispatcher classes.
Add the code shown below in bold to the Main method, immediately after the statement that opens the ServiceHost object:
```
 host.Open(); ChannelDispatcher dispatcher =     (ChannelDispatcher)host.ChannelDispatchers[0]; ServiceThrottle throttle = dispatcher.ServiceThrottle; if (throttle == null)     Console.WriteLine("Service is using default throttling behavior"); else     Console.WriteLine("Instances: {0}\nCalls: {1}\nSessions: {2}",         throttle.MaxConcurrentInstances, throttle.MaxConcurrentCalls,         throttle.MaxConcurrentSessions); Console.WriteLine("Service running");
```
This code retrieves a reference to the ChannelDispatcher object used by the service (the service has only a single binding, so the WCF runtime creates only a single ChannelDispatcher when the service opens). The code then examines the ServiceThrottle property of this ChannelDispatcher object. If it is null, then the administrator or developer has not specified any customized throttling settings so the service uses the default values. If the ServiceThrottle property is not null, then the service is using a customized throttling behavior and the values provided by the administrator or developer are displayed.
Start the solution without debugging. In the service console window, notice that the service is using the default throttling behavior.

Press Enter in the client console window.

The client console window displays ten messages of the form “Client n: 1st AddItemToCart,” where n is the number identifying the instance of the client. In the service console window, notice that the message “AddItemToCart operation started” appears ten times in succession, as shown in the following image:

This indicates that the service is handling all ten clients simultaneously. As each method completes, the service displays “AddItemToCart operation completed” messages and the clients invoke further operations. At this point, the output becomes a little more chaotic, but the important point is that in this “unthrottled” state the service has not prevented any of the ten clients from invoking operations at the same time (the default value for the maximum number of concurrent calls is greater than 10).

When the clients have finished (the message “Client n: Goods Purchased” has appeared ten times), press Enter to close the client console window. Press Enter to close the service console window.
In Visual Studio 2005, edit the App.config file in the ShoppingCartHost project by using the WCF Service Configuration Editor.
In the WCF Service Configuration Editor, expand the Advanced Folder and then click the Service Behaviors folder. In the right pane, click the New Service Behavior Configuration link.

A new behavior appears in the right pane. Change the Name property of this behavior to ThrottleBehavior.
In the right pane, click Add and add a serviceThrottling element to the behavior.
Double-click the serviceThrottling behavior element you have just added.

The properties for this element appear: MaxConcurrentCalls, MaxConcurrentInstances, and MaxConcurrentSessions. Each property displays its default value:
Change the value of the MaxConcurrentCalls property to 3.
In the left pane, click the ShoppingCartService.ShoppingCartServiceImpl node in the Services folder. In the right pane, set the BehaviorConfiguration property to ThrottleBehavior.
Save the configuration file, but leave the WCF Service Configuration Editor open.

In Visual Studio 2005, start the solution without debugging. In the service console window, you should see that the service is now using the throttling behavior you have just defined rather than the default settings.

Press Enter in the client console window.

In the client console window, all ten clients output the message “Client n:1st AddItemToCart,” but the service console shows something different from before; initially only three “AddItemToCart operation started” messages appear. This is because the service now supports only three concurrent operation calls. Each request that occurs after this is queued by the ChannelDispatcher. As each call finishes, displaying “AddItemToCart operation completed,” the ChannelDispatcher releases the next request from its queue, and you see the message “AddItemToCart operation started” appear for the next client. Thereafter, each time an operation completes, the ChannelDispatcher releases the next request. You should see alternating “completed” and “started” messages until the clients have finished their work, as the time taken for the service to process each request is at least 5 seconds, and this is longer than the time taken for each client to send the next request when the previous one completes.

Note

The ChannelDispatcher releases requests from its queue on a first-come first-served basis. Currently, WCF does not allow you to specify that the requests for one client should have a higher priority than another.

Press Enter to close the client console window, and then press Enter to close the service console window.

Return to the WCF Service Configuration Editor and click the serviceThrottling service behavior element in the left pane. In the right pane, increase the MaxConcurrentCalls property to 16 (the default value) and set the MaxConcurrentSessions property to 3. Save the configuration file, but leave the WCF Service Configuration Editor open.

In Visual Studio 2005, start the solution without debugging. In the service console window, you should see that the service is using the updated throttling behavior.

Press Enter in the client console window.

Again, in the client console window, all ten clients output the message “Client n: 1st AddItemToCart,” and the service console window shows three calls to the AddItemToCart operation starting and completing. However, when these calls complete, if you observe the messages in the client console window, you will see that only clients 0, 1, and 2 invoke the AddItemToCart operation the second time; the other clients are held pending by the ChannelDispatcher because it has reached the maximum number of concurrent sessions it will allow. Clients 0, 1, and 2 complete their cycle of work calling AddItemToCart a third time, followed by the GetShoppingCart operation and the Checkout operation. Only when Checkout completes and a client closes its session before terminating does the next client actually get to continue. You should see messages occurring in batches of three in the client console window (three “2nd AddItemToCart” messages, three “3rd AddItemToCart” messages, and so on), as each set of three sessions executes.

Some of the later sessions might report the exception “The operation did not complete within the allotted time-out of 00:01:00. …,” as the time between them submitting the initial AddItemToCart request and the service allowing that request to be handled exceeds the time-out limit specified for the client binding.

Note

Although the ChannelDispatcher queues the requests to create each session in the order it receives them, once a session starts running, there is no guarantee that it will be serviced before or after any other running session in this example. For example, when sessions for clients 3, 4, and 5 are running, you might see messages indicating that operations for client 4 execute before those of client 3; this is due to the scheduling algorithm in the operating system deciding when to execute each thread in the client and in the service.

Press Enter to close the client console window, and then press Enter to close the service console window.

Return to the WCF Service Configuration Editor. In the right pane, set the MaxConcurrentSessions property to 10 (the default value). Save the configuration file, and close the WCF Service Configuration Editor.

This exercise has shown you the effects of using service throttling to control the maximum number of concurrent calls and sessions that a service will permit. To what values should you set the service throttling properties? I am afraid that there is no simple answer. You need to test your service against a realistic workload and observe whether client applications are blocked for extended periods. Remember that the purpose of service throttling is to prevent your service from being inundated with a flood of requests that it does not have the resources to cope with. You should set the service throttling properties to ensure that, when a client request is accepted and execution actually starts, the computer hosting the service has sufficient resources available to be able to complete the operation before the client times out, as this would hinder overall performance further. In a transactional environment, aborted client requests generate even more work for the service, as it then has to rollback all the transactional work it has performed.

WCF and Service Instance Pooling

The WCF runtime creates service instances to handle client requests. If the service is using the PerSession instance context mode, the instance can last for several operations. If the service is using the PerCall instance context mode, each operation call results in a new service instance, which is discarded and destroyed when the operation ends. Creating and destroying instances are expensive, potentially time-consuming tasks. Service instance pooling would be very useful in this scenario.

When using instance pooling, the WCF runtime would create a pool of service instance objects when the service starts. When using the PerCall instance context mode, as client applications invoke operations, the WCF runtime would retrieve a pre-created service instance from the pool and return it to the pool when the operation completes. When using the PerSession instance context mode, the same semantics apply, but the WCF runtime would obtain a service instance from the pool at the start of the session and return it at the end of the session. For security purposes, any data held by the service instance (fields in the class defining the service implementation) would be cleared as the instance was returned to the pool.

As you might have gathered from the tone of the previous paragraph, WCF does not provide service instance pooling directly, but it is possible to extend WCF by defining your own custom behavior that implements pooling. WCF supplies the IInstanceProvider interface in the System.ServiceModel.Dispatcher namespace that you can use to define your own service instance dispatch mechanism. This is a useful technique, but the details are outside the scope of this book, although Chapter 10 provides an example of how to implement a service behavior. For more information, see the topic “Pooling” in the Microsoft Windows SDK Documentation. This topic is also available on the Microsoft Web site at http://msdn2.microsoft.com/en-gb/library/ms751482(en-us).aspx.