Interpreting Test Results

You now have the data in whichever format you wanted it, be that in a graph or in a CSV file. What are you looking for? The two simplest measures are mean and standard deviation.

Examining the Mean

The mean is the average response time for a sample. It indicates the central tendency of your server’s performance. But what about variation? If your server has a good mean response time but a huge amount of variation among the samples, you’ll end up with the average user having a decent experience and the occasional user or request having a really bad (or really good) experience. It’s this kind of variation that drives an unpredictable and frustrating user experience. A good measure of variation is standard deviation.

Examining the Standard Deviation

Standard deviation is a measure of how widely values in a sample are dispersed from the mean value. Hence, a higher standard deviation indicates more variance in response time. You have many ways to splice and dice performance data. Sometimes, mentally comparing two samples is enough. Other times, it may make sense to delve into more complex statistical tools, such as regression analysis, for predictive modeling of a process’s performance. Although a description of these statistical methods is beyond the scope of this book, you can find many online and printed statistics resources at http://www.prndata.com/statistics_sites.htm.