8.5 Quality of Service

---

8.5 Quality of Service

The Quality of Service component in IBM Tivoli Monitoring for Transaction Performance Version 5.2 samples data from real-time, live HTTP transactions against a Web server and measures, among other items, the time required for the round trip of each transaction. The Quality of Service component measurements include:

• User Experience Time

• Back End Service Time

• Page Render Time

To gather this type of information, QoS intercepts the communication between end users and Web servers by means of reverse-proxy technology. This allows QoS to measure response times and to manage ARM correlators. The use of ARM allows QoS to scale better and to be incorporated with other measurement technologies, such as J2EE and STI.

When a HTTP request reaches QoS, QoS checks the request to see if the HTTP headers contain an ARM correlator from a parent transaction. If a correlator is discovered, it will consider itself to be a non-edge application (a subtransaction) in relation to gathering and recording ARM data. In case of the absence of a correlator, QoS will consider itself to be the edge application for this transaction, and generate a correlator, which is included in the HTTP request as it is passed on the server that hosts the called application.

The reverse proxy implementation provides a single entry-point to several Web servers much like a normal proxy works as an Internet gateway for multiple workstations on a corporate network, as depicted in Figure 8-17 on page 258. Without the reverse proxy, the IP addresses of all the Web servers has to be known by the requestors. With the reverse proxy, the requestors only need to know the IP address of the reverse proxy.

Figure 8-17: Proxies in an Internet environment

This technology is primarily implemented to circumvent some of the shortcomings of the TCP/IP addressing schema by removing the need for all servers and workstations to be addressable (known) to all other systems on the Internet, which also may be regarded as an additional security feature.

When working with the Quality of Service monitoring component, you should be familiar with the following terms:

Origin server	The Web server that you want to monitor.
Proxy server	A virtual server (implemented at the origin server or on a remote computer) that acts as a gateway to specific Web Servers. Normally, transactions within a Web server measures the time required to complete the transaction. This virtual server runs within IBM HTTP Server Version 1.3.26.1, which comes with the QoS monitoring component.
Reverse proxy	A physical HTTP Server that hosts the virtual proxy servers pointing to the origin servers. The reverse proxy server also hosts the QoS monitoring component. The reverse proxy server may be installed directly on the origin server or on a remote computer. Running QoS on the same machine as the origin server may be beneficial, because it eliminates network issues (speed, delay, collisions, and bandwidth).
Digital certificates	Authentication documents that secure communications for Quality of Service monitoring.

8.5.1 QoS Component deployment

To deploy the Quality of Service component to a Management Agent, follow the steps below:

1. From the home page of the Management Server console, click on System Administration → Work with Agents. The Work with Agents dialog depicted in Figure 8-18 will be displayed.

   
   Figure 8-18: Work with agents QoS

2. Select the target to which QoS is to be deployed, and select the Deploy Quality of Service component from the action selection drop-down menu at the top of the Work with Agents dialog. Click Go to go to the configuration of the new Quality of Service component.

   The Deploy Components and/or Monitoring Component dialog shown in Figure 8-19 is used to configure the parameters for the QoS component. The information to be provided is grouped in two Server Configuration sections:

   HTTP Proxy	Specifies the networking parameters for the virtual server that will receive the requests for the origin server. The host name should be that of the Management Agent, which is the target of the QoS deployment, and the port number can be set to any free port on that system.
   Origin HTTP Proxy	Specifies the networking parameters of the origin server, which will serve the requests forwarded from the virtual server residing on the QoS system. The host name should be set to the name of the system hosting the application server (for example, WebSphere Application Server), and the port number should be set to the port that the application server listens to for a particular application.

   
   Figure 8-19: Deploy QoS components

   Provide the values as they apply to your environment, and click OK to start the deployment. After couple of minutes the Management Agent will be rebooted and the Quality of Service component has been deployed.

3. To verify that the installation was successful, refresh the Work with Agents dialog, and verify that the status for the QoS Component on the Management Agent in question shows Installed, as shown in Figure 8-20.

   
   Figure 8-20: Work with Agents- QoS installed

8.5.2 Creating discovery policies for QoS

The purpose of the QoS discovery policy is to gather information about the URIs that are handled by the QoS Agent. As is the case for STI Agents, the URIs have to be discovered before monitoring policies can be defined and deployed. The Quality of Service discovery policy returns URIs only from Management Agents on which a Quality of Service listener is deployed.

Note 

Please remember that specific discovery policies has to be created for each type of agent: QoS, J2EE, and STI.

Before setting up any policies for a QoS Agent, it is important to understand the concept of virtual servers.

The term virtual server refers to the practice of maintaining more than one server on one machine. These Web servers may be differentiated by IP, host name, and/or port number.

QoS and virtual servers

Even though the GUI for QoS configuration does not allow for defining multiple origin-server/virtual-server pairs, there is a way to use one QoS machine to measure requests for several back-end Web servers.

The advantage to this setup is that only one machine is used to measure the transactions' response times of a number of machines that do real work. However, one disadvantage of this setup is that the QoS system introduces a potential bottleneck and a single-point-of-failure. Another disadvantage is that there is no distinction in the metrics measured for the different servers, as the base for the distinguishing where the metrics come from is the QoS and not the back-end Web servers.

To set up a single QoS Agent to measure multiple back-end servers, please understand that because the QoS acts as a front end for the back-end Web server, the browsers connect to the QoS rather than to the Web server. If the QoS is to act as a front-end for different servers, it must have a separate identity for each server it serves as a front end for. To define separate identities, a virtual host has to be defined in the QoS HTTP server for each back-end server. These virtual servers may be either address- or name-based:

Address-based	The QoS has multiple IP addresses and multiple network interfaces, each with its own host name.
Name-based	The QoS has multiple host names pointing to the same IP address.

Both ways imply that the DNS server must be aware that the QoS has multiple identities.

Definitions of virtual servers are, after initial deployment of the Quality of Service component, performed by manually editing the HTTP configuration file on the QoS system. Example 8-2 shows an HTTP configuration file (http.conf) for a QoS system named tivlab01(9.3.5.14), which the alias of tivlab02(9.3.5.14), which is configured to use the default HTTP port (80). It has two virtual servers, backend1 and backend2, which in turn reverse proxy the hosts at 9.3.5.20 and 9.3.5.15.

Example 8-2: Virtual host configuration for QoS monitoring multiple application servers

 # This is for name-based virtual host support. NameVirtualHost backend1:80 NameVirtualHost backend2:80 # For clarity, place all listen directives here. Listen 9.3.5.14:80 # This is the main virtual host created by install.       # ########################################################### <VirtualHost backend1:80> #SSLEnable ServerName backend1 QoSMContactURL http://9.3.5.14:80/ # Enable the URL rewriting engine and proxy module without caching. RewriteEngine on RewriteLogLevel 0 ProxyRequests on NoCache * # Define a rewriting map with value-lists. #             mapname key: filename #RewriteMap    server "txt:<QOSBASEDIR>/IBMHTTPServer/conf/apache-rproxy.conf-servers" # Make sure the status page is handled locally and make sure no one uses our # proxy except ourself. RewriteRule    ^/apache-rproxy-status.*  -  [L] RewriteRule    ^(https|http|ftp)://.*    -  [F] # Now choose the possible servers for particular URL types. RewriteRule    ^/(.*\.(cgi|shtml))$  to://9.3.5.20:80/$1  [S=1] RewriteRule    ^/(.*)$               to://9.3.5.20:80/$1 # ... and delegate the generated URL by passing it through the proxy module RewriteRule    ^to://([^/]+)/(.*)    http://$1/$2   [E=SERVER:$1,P,L] # ... and make really sure all other stuff is forbidden when it should survive # the above rules. RewriteRule    .*                    -              [F] # Setup URL reverse mapping for redirect reponses. ProxyPassReverse    /    http://9.3.5.20:80/ ProxyPassReverse    /    http://9.3.5.20/ </VirtualHost> ########################################################### # second backend machine created manually ########################################################### <VirtualHost backend2:80> #SSLEnable ServerName backend2 QoSMContactURL http://9.3.5.14:80/ # Enable the URL rewriting engine and proxy module without caching. RewriteEngine on RewriteLogLevel 0 ProxyRequests on NoCache * # Define a rewriting map with value-lists. #             mapname key: filename #RewriteMap    server "txt:<QOSBASEDIR>/IBMHTTPServer/conf/apache-rproxy.conf-servers" # Make sure the status page is handled locally and make sure no one uses our # proxy except ourself. RewriteRule    ^/apache-rproxy-status.*  -  [L] RewriteRule    ^(https|http|ftp)://.*    -  [F] # Now choose the possible servers for particular URL types. RewriteRule    ^/(.*\.(cgi|shtml))$  to://9.3.5.15:80/$1  [S=1] RewriteRule    ^/(.*)$               to://9.3.5.15:80/$1 # ... and delegate the generated URL by passing it through the proxy module RewriteRule    ^to://([^/]+)/(.*)    http://$1/$2   [E=SERVER:$1,P,L] # ... and make really sure all other stuff is forbidden when it should survive # the above rules. RewriteRule    .*                    -              [F] # Setup URL reverse mapping for redirect reponses. ProxyPassReverse    /    http://9.3.5.15:80/ ProxyPassReverse    /    http://9.3.5.15/ </VirtualHost> 

In a live production environment, chances are that multiple QoS systems will be used to monitor a variety of application servers hosting different applications, as depicted in Figure 8-21 on page 265.

Figure 8-21: Multiple QoS systems measuring multiple sites

When planning to use multiple virtual servers on a single or multiple QoS system(s), please take the following into consideration:

Policy creation	When scheduling a policy against particular end points, it makes sense to schedule it against groups that are created and maintained as virtual hosts. A customer that want to schedule a job against www.telia.com:80, for example, would want to select the group with all of the above QoS systems. When scheduling a policy against www.kal.telia.com:85, however, a group only contains QoS1. The name of the server QoS1 in this case does not give the user/customer any indication of what virtual hosts exist on each machine.
Endpoint Groups	Endpoint Groups are an obvious match for this needed functionality. It is possible to name a group with the appropriate virtual host string (www.telia.com:80, for example).
Modification of Endpoint Groups for QoS Virtual Hosts	An extra flag will be added to the Object Model definition of an Endpoint Group to allow you to determine if each specific Endpoint Group is a virtual host. It will be a Boolean value for use by UI and the object model itself
Implications for UI	The UI will need to only allow the scheduling of QoS policies against an Endpoint Group that is also a virtual host. The UI as well will need to not allow any editing/modification of Endpoint Groups that are virtual hosts; this will be handled by the QoS behavior on the Management Agents.
Update Mechanism	Virtual hosts will be detected by the QoS component on each Management Agent. When the main QoS service is started on the Management Agent, a script will run, which will detect the virtual hosts installed on the particular Management Agent. Messages will then be sent to the Management Server; a Web service will be created on the Management Server as an interface to the session beans that will create, edit, and otherwise manage the endpoint groups that are virtual hosts.

Please consult the manual IBM Tivoli Monitoring for Transaction Performance User's Guide Version 5.2.0, SC32-1386 for more details.

Create discovery policies for QoS

Before creating a discovery policy for Quality of Service, you should note that QoS listening policies may be executed without prior discovery. However, if you do not know which areas of your Web environment require monitoring, create and run a discovery policy first and then create a listening policy.

To create a a QoS discovery policy for the home page of the TMTP Version 5.2 Console, select Configuration → Work with Discovery Policies. This will make the Work with Discovery Policies dialog shown in Figure 8-22 on page 267 appear.

Figure 8-22: Work with discovery policies

To create a new policy, you should perform the following steps:

1. Select the QoS type of discovery policy, and click Create New, which will bring up the Configure QoS Listener dialog shown in Figure 8-23 on page 268.

   
   Figure 8-23: Configure QoS discovery policy

2. Add your URI filters and provide sampling information. Click Next to proceed to choose a schedule in the Work with Schedules dialog shown in Figure 8-24 on page 269.

   
   Figure 8-24: Choose schedule for QoS

3. Select a schedule, or create a new one that will suit your needs. Click Next to continue with Agent Group selection, as shown in Figure 8-25 on page 270.

   
   Figure 8-25: Selecting Agent Group for QoS discovery policy deployment

4. Before performing the final step, you have to select the group(s) of QoS Agents that the newly created QoS discovery policy will be distributed to. Select the appropriate group(s), and click Next.

5. Finally you have to provide a name. In this case trade_qos-dis is used. Also, determine if the profile is to be sent to the agents in the Agent Group(s) immediately, or wait until the next scheduled distribution. Click Finish to save the definition of the Quality of Service discovery profile (see Figure 8-26 on page 271).

   
   Figure 8-26: Assign name to new QoS discovery policy

Create a listening policy for QoS

The newly created discovery profile may be used as the starting point for creating the QoS listening policy (the one that actually collects and reports on transaction performance data). This will allow you to select transactions that have been discovered as the basis for the listening policy. Listening policies may also be created directly without the use of previously discovered transactions.

To create a listening policy by using the data gathered by the discovery policy, start by going to the home page of the TMTP Version 5.2 console and use the left side navigation pane to select Configuration → Work with Discovery Policies. The Work with Discovery Policies dialog shown in Figure 8-27 on page 272 will be displayed.

Figure 8-27: View discovered transactions to define QoS listening policy

Now, perform the following:

1. Select QoS and the desired type of policie(s) (QoS or J2EE) from the drop-down list at the top of the dialog.

2. Select the appropriate discovery policies. In our example, only trade_qos_dis was selected.

3. Select View Discovered Transactions from the drop-down list just above the list of discovery profiles and press Go. This will display a list of discovered transactions in the View Discovered Transactions, as shown in Figure 8-28 on page 273.

   
   Figure 8-28: View discovered transaction of trade application

4. From the View Discovered Transactions dialog, select the transaction that will be the basis for the listening policy:

   1. Select a transaction.

   2. Select Create Component Policy From in the function drop-down menu at the top of the transaction list.

   3. Click Go.

   This will take you to the Configure QoS Listener dialog shown in Figure 8-29 on page 274.

   
   Figure 8-29: Configure QoS set data filter- write data

5. Apply appropriate values for filtering your data.

   You can apply filters that will help you collect transaction data from requests that originate from specific systems (IP addresses) or groups thereof. The filtering may be defined as a regular expression.

   In addition, you should specify how much data you want to capture per minute, and whether or not instance data should be stored along with the aggregated values. In case a threshold (which you will specify in the following dialog) is violated, TMTP Version 5.2 will automatically collect instance data for a number of invocations of the same transaction. You can customize this number to provide the level of detail needed in your particular circumstances.

   Click Next to go on to defining thresholds for the listening policy.

6. The Configure QoS Settings dialog, shown in Figure 8-30 on page 275, is used to define global values for threshold and event processing in QoS.

   
   Figure 8-30: Configure QoS automatic threshold

   To create a specific threshold, select the type in the drop-down menu under the dialog heading. Two types are available:

   • Performance

   • Transaction Status

   When clicking Create, the Configure QoS Thresholds dialog shown in Figure 8-31 on page 276 will be displayed.

   
   Figure 8-31: Configure QoS automatic threshold for Back-End Service Time

   Detailed descriptions of each of the properties are available in the IBM Tivoli Monitoring for Transaction Performance User's Guide Version 5.2.0, SC32-1386.

7. In the Configure QoS Thresholds, you can specify thresholds specific to each of the types chosen in the previous dialog.

   A Quality of Service transaction status threshold is used to detect a failure of the monitored transaction, or to detect the receipt of an HTTP response code from the Web server, or specific response times related to the QoS transaction during monitoring. Violation events are generated, or triggered, when failure occurs or when a specified HTTP response code is received. Recovery events and the associated notification are generated when the transaction executes as expected after a violation.

   Based on your selection, you can set thresholds for the following:

   Performance	Back-End Service Time Page Render Time Round Trip Time
   Transaction Status	Failure or specific HTTP return codes

   For each threshold you are creating, you should press Apply to save your settings, and when finished, click Next to continue to the Configure J2EE Settings dialog.

8. Since this does not provide functions for the QoS listening policy, click Next again to proceed to the schedule selection for the policy.

9. Schedules for Quality of Service listening policies are selected the same way as for any other policy. Please refer to 8.4.4, "Playback schedule definition" on page 248 for more details related to schedules. Click Next to go on to select Agent Groups for the listening policy.

10. Agent Group selection is common to all policy types. Please refer to the description provided in item 4 on page 270 for further details. Click Next to finalize your policy definition.

11. Having defined all the necessary properties of the QoS listening policy, all that is left before you can save and deploy the listening policy is to assign a name, and determine when to deploy the newly defined listening policy to the Management Agents.

From the Assign Name dialog shown in Figure 8-32, select your preferred distribution time and click Finish.

Figure 8-32: Configure QoS and assign name

---