Section 4.7. Throttling

4.7. Throttling

While it is not a direct instance management technique, throttling enables you to restrain client connections and the load they place on your service. Throttling enables you to avoid maxing out your service and the underlying resources it allocates and uses. When throttling is engaged, if the settings you configure are exceeded, WCF will automatically place the pending callers in a queue and serve them out of the queue in order. If the client's call timeout expires while pending in the queue, the client will get a TimeoutException. Throttling is done per service type; that is, it affects all instances of the service and all its endpoints. This is done by associating the throttle with every channel dispatcher the service uses.

WCF allows you to control some or all of the following service consumption parameters:

• Maximum number of concurrent sessionsthe overall number of outstanding clients that have a session at the transport level with the service. In plain terms, this number means the maximum overall number of outstanding clients using TCP, IPC, or any of the WS bindings with sessions. When using the basic binding or any of the WS bindings without a transport session, this number has no effect because of the connectionless nature of a basic HTTP connection. The default value is 10.

• Maximum number of concurrent callsthe total number of calls currently in progress across all service instances. This number should be kept usually at 1 to 3 percent of the maximum number of concurrent sessions. The default value is set to 16.

• Maximum number of concurrent instancesthis number actually stands for the total number of contexts concurrently alive. The default value is unlimited. How instances map to contexts is a product of the instance context management mode as well as context and instance deactivation. With a per-session service, the maximum number of instances is both the total number of concurrently active instances and the total number of concurrent sessions. When instance deactivation is employed, there could be far fewer instances than contexts, and yet clients will be blocked if the number of contexts has reached the maximum number of concurrent instances. With a per-call service, the number of instances is actually the same as the number of concurrent calls. Consequently, the maximum number of instances with a per-call service is the lesser of the maximum concurrent instances and the maximum concurrent calls. The value of maximum concurrent instances is ignored with a singleton service since it can only have a single instance anyway.

Throttling is a hosting and deployment aspect. When you design a service, make no assumptions about throttling configurationalways assume your service will bear the full brunt of the client's load. This is why although it is fairly easy to write a throttling behavior attribute, WCF does not offer one.

4.7.1. Configuring Throttling

Throttling is typically configured by administrators in the config file. This enables you to throttle the same service code differently over time or across deployment sites. The host can also programmatically configure throttling based on some runtime decisions.

4.7.1.1. Administrative throttling

Example 4-9 shows how to configure throttling in the host config file. Using the behaviorConfiguration tag, you add to your service a custom behavior that sets throttled values.

Example 4-9. Administrative throttling

 <system.serviceModel>    <services>       <service name = "MyService" behaviorConfiguration = "ThrottledBehavior">          ...       </service>    </services>    <behaviors>       <serviceBehaviors>          <behavior name = "ThrottledBehavior">             <serviceThrottling                maxConcurrentCalls     = "12"                maxConcurrentSessions  = "34"                maxConcurrentInstances = "56"             />          </behavior>       </serviceBehaviors>    </behaviors> </system.serviceModel> 

4.7.1.2. Programmatic throttling

The host process can programmatically throttle the service based on some runtime parameters. You can only do so before the host is opened. Although the host can override the throttling behavior found in the config file by removing it and adding its own, you typically should provide a programmatic throttling behavior only when there is no throttling behavior in the config file.

The ServiceHostBase class offers the Description property of the type ServiceDescription:

 public abstract class ServiceHostBase : ... {    public ServiceDescription Description    {get;}    //More members } 

The service description, as its name implies, is the description of the service with all its aspects and behaviors. ServiceDescription contains a property called Behaviors of the type KeyedByTypeCollection<I>, with IServiceBehavior as the generic parameter.

Example 4-10 shows how to set the throttled behavior programmatically.

Example 4-10. Programmatic throttling

 ServiceHost host = new ServiceHost(typeof(MyService)); ServiceThrottlingBehavior throttle; throttle = host.Description.Behaviors.Find<ServiceThrottlingBehavior>( ); if(throttle == null) {    throttle = new ServiceThrottlingBehavior( );    throttle.MaxConcurrentCalls     = 12;    throttle.MaxConcurrentSessions  = 34;    throttle.MaxConcurrentInstances = 56;    host.Description.Behaviors.Add(throttle); } host.Open( ); 

First the hosting code verifies that no service throttling behavior was provided in the config file. This is done by calling the Find<T>( ) method of KeyedByTypeCollection<I> using ServiceThrottlingBehavior as the type parameter.

ServiceThrottlingBehavior is defined in the System.ServiceModel.Design namespace:

 public class ServiceThrottlingBehavior : IServiceBehavior {    public int MaxConcurrentCalls    {get;set;}    public int MaxConcurrentSessions    {get;set;}    public int MaxConcurrentInstances    {get;set;}    //More members } 

If the returned throttle is null, the hosting code creates a new ServiceThrottlingBehavior, sets its values, and adds it to the behaviors in the service description.

4.7.1.3. Streamlining with ServiceHost<T>

You can extend ServiceHost<T> to automate the code in Example 4-10, as shown in Example 4-11.

Example 4-11. Extending ServiceHost<T> to handle throttling

 public class ServiceHost<T> : ServiceHost {    public void SetThrottle(int maxCalls,int maxSessions,int maxInstances)    {       ServiceThrottlingBehavior throttle = new ServiceThrottlingBehavior( );       throttle.MaxConcurrentCalls = maxCalls;       throttle.MaxConcurrentSessions = maxSessions;       throttle.MaxConcurrentInstances = maxInstances;       SetThrottle(throttle);    }    public void SetThrottle(ServiceThrottlingBehavior serviceThrottle)    {       SetThrottle(serviceThrottle,false);    }    public void SetThrottle(ServiceThrottlingBehavior serviceThrottle,                            bool overrideConfig)    {       if(State == CommunicationState.Opened)       {          throw new InvalidOperationException("Host is already opened");       }       ServiceThrottlingBehavior throttle =                            Description.Behaviors.Find<ServiceThrottlingBehavior>( );       if(throttle != null && overrideConfig == false)       {          return;       }       if(throttle != null) //overrideConfig == true, remove the configured one       {          Description.Behaviors.Remove(throttle);       }       if(throttle == null)       {          Description.Behaviors.Add(serviceThrottle);       }    }    public ServiceThrottlingBehavior ThrottleBehavior    {       get       {          return Description.Behaviors.Find<ServiceThrottlingBehavior>( );       }    }    //More members } 

ServiceHost<T> offers the SetThrottle( ) method, which accepts the throttle to use, as well as a Boolean flag indicating whether or not to override the configured values, if present. The default value (using an overloaded version of SetThrottle( )) is false. SetThrottle( ) verifies that the host has not been opened yet using the State property of the CommunicationObject base class. If it is required to override the configured throttle, SetThrottle( ) removes it from the description. The rest of Example 4-11 is similar to Example 4-10. Here is how you can use ServiceHost<T> to set a throttle programmatically:

 ServiceHost<MyService> host = new ServiceHost<MyService>( ); host.SetThrottle(12,34,56); host.Open( ); 

In a similar manner, InProcFactory<T> presented in Chapter 1 is also extended to streamline throttling.

4.7.1.4. Reading throttled values

The throttled values can be read at runtime by service developers for diagnostic and analytical purposes. At runtime, the service instance can access its throttled properties from its dispatcher. First, obtain a reference to the host from the operation context. The host base class ServiceHostBase offers the read-only ChannelDispatchers property:

 public abstract class ServiceHostBase : CommunicationObject,... {    public ChannelDispatcherCollection ChannelDispatchers    {get;}    //More members } 

ChannelDispatchers is a strongly typed collection of ChannelDispatcherBase objects:

 public class ChannelDispatcherCollection :                                       SynchronizedCollection<ChannelDispatcherBase> {...} 

Each item in the collection is of the type ChannelDispatcher. ChannelDispatcher offers the property ServiceThrottle:

 public class ChannelDispatcher : ChannelDispatcherBase {    public ServiceThrottle ServiceThrottle    {get;set;}    //More members } public sealed class ServiceThrottle {    public int MaxConcurrentCalls    {get;set;}    public int MaxConcurrentSessions    {get;set;}    public int MaxConcurrentInstances    {get;set;} } 

ServiceThrottle contains the configured throttled values:

 class MyService : ... {    public void MyMethod( ) //Contract operation    {       ChannelDispatcher dispatcher = OperationContext.Current.                                    Host.ChannelDispatchers[0] as ChannelDispatcher;       ServiceThrottle serviceThrottle = dispatcher.ServiceThrottle;       Trace.WriteLine("Max Calls = " + serviceThrottle.MaxConcurrentCalls);       Trace.WriteLine("Max Sessions = " + serviceThrottle.MaxConcurrentSessions);       Trace.WriteLine("Max Instances = " + serviceThrottle.MaxConcurrentInstances);    } } 

Note that the service can only read the throttled values and has no way of affecting them. If the service tries to set throttled values, it will get an InvalidOperationException.

Again, you can streamline the throttle lookup via ServiceHost<T>. First, add a ServiceThrottle property:

 public class ServiceHost<T> : ServiceHost {    public ServiceThrottle Throttle    {       get       {          if(State == CommunicationState.Created)          {             throw new InvalidOperationException("Host is not opened");          }          ChannelDispatcher dispatcher = OperationContext.Current.                                    Host.ChannelDispatchers[0] as ChannelDispatcher;          return dispatcher.ServiceThrottle;       }    }    //More members } 

Then, use ServiceHost<T> to host the service and use the ServiceThrottle property to access the configured throttle:

 //Hosting code ServiceHost<MyService> host = new ServiceHost<MyService>( ); host.Open( ); class MyService : ... {    public void MyMethod( )//Contract operation    {      ServiceHost<MyService> host = OperationContext.Current.                                                     Host as ServiceHost<MyService>;      ServiceThrottle serviceThrottle = host.Throttle;      ...    } } 

You can only access the Throttle property of ServiceHost<T> after the host is opened. This is because the dispatcher collection is initialized only after opening the host.

4.7.2. Throttled Connections in the Binding

When you use the TCP and named-pipe bindings, you can also configure the maximum connection number for a particular endpoint in the binding itself. Both the NetTcpBinding and the NetNamedPipeBinding offer the MaxConnections property:

 public class NetTcpBinding : Binding,... {    public int MaxConnections    {get;set;} } public class NetNamedPipeBinding : Binding,... {    public int MaxConnections    {get;set;} } 

On the host side, you can set that property either programmatically or by using a config file:

 <bindings>    <netTcpBinding>       <binding name = "TCPThrottle" maxConnections = "25"/>    </netTcpBinding> </bindings> 

MaxConnections defaults to 10. When both a binding-level throttle and a service-behavior throttle sets the max connection value, WCF chooses the lesser of the two.