Analyzing the Test Results

The logs show a lot of detail that can be used to create accurate reports. Client usage logs generated on the servers (like the one shown in figure 17.5) can be compiled and used to create detailed client demographic reports, for example. Commercial Web sites can base customer billing on the length of time clients stay tuned to a stream or the number of hits a file gets.

At Fabrikam, client logs can be used by the IT department to locate streaming errors. If the cause of an error isn’t obvious, they can use the times on the client log to reference a performance log.

Each record in the client log corresponds to an event from the client. The log in figure 17.5 shows a few seconds of a Windows Media Load Simulator run. Client logs from simulations will not always give you useful information because the simulator does not always provide the server with the same information an actual Player would provide. But you can glance through it for large blocks of streaming errors, for example.

Figure 17.5: Client usage logs provide details about streaming activity on the server.

If you note a problem at a given time, such as a large number of playback errors, you could cross-check logs created by the performance monitors for each server. If you looked at the performance log for that time, you might note a high processor time and late reads value due to a high number of concurrent streaming players. If there was nothing unusual on the server side at that time other than an unusually high number of clients, you might suspect the network or a configuration problem.

You could then check the Windows Media Load Simulator WMLog and look for clients that reported a high number of failures. Figure 17.4 shows the data for one client in the log. The first part of the log records the error activity that printed out in real time during the simulation run on the Load Simulation Status tab on Windows Media Load Simulator. If a client reported failure to open, configuration was correct, and you knew the server was not under stress, then you might suspect a slow network or a stressed client computer.

The Windows Media Load Simulator tests have several purposes. First, they enable you to pick up any gross configuration errors in the server, try different server settings, and learn how the server operates. Simulations can also be used to configure limits on the server. Figure 17.6 shows the Limits list on the Properties tab of a server. You can set limits for the entire server or individual publishing points. If your computer and network are slow, try limiting the number of player connections or connection rate. If you need to preserve network bandwidth, limit aggregate player bandwidth.

Figure 17.6: Configuring usage limits for a server.

The Fabrikam server farm itself can most likely handle more processing and streaming than the network. However, if you are running Windows Media Services on a slow computer or if network bandwidth is an issue, you can place limits on bandwidth and the number of client connections. When deciding what numbers to use, keep in mind that it requires less bandwidth to stream a few high-bit-rate streams than it does to stream many low-bit-rate streams, even though the total bit rate is the same. This is because each connection requires a certain amount of bandwidth overhead.

Delivering a large number of streams also requires more processing power and reads from the hard drive. This is also true of short files and long files. As you can see with the simulation, playing many short files requires more opening and closing of sessions and seeking, which requires more bandwidth and CPU time than would long files. Try simulations against many short files, and then try the simulation again with a few long files. Also try adding clients that just seek, open, and close files.

Try to estimate what the typical and peak mix of content and playback behavior will be in your enterprise. It will only be an estimate, but this will provide testing results that are more applicable to your particular scenario than simply entering random client profile numbers. The lab test is the first of enterprise performance baselines. In a production environment, more tests should be done during deployment to make sure the lab results are accurate.