After Testing Is Under Way

Suppose that you are in the middle of your test plan when you realize that there is a completely different set of experiments that should be run. Should you walk away from your well-designed plan? Should you only add some new tests to the end of the plan and continue on the original path?

Creativity is greatly underestimated during performance testing and tuning; the best course is one of flexibility within certain constraints. Make careful note of your insights and ideas during this process write them down as soon as they appear. There's no need to scrap all of the work you devoted to creating your test plan and performing your experiments. These new ideas can serve as a valuable addition to your well-designed plan.

For example, take a look at one of the tests from the preceding listing, along with the new test that it spawned:

 Test #:            204 Date:              6-October-2005 Start time:        09:29 Finish time:       09:47 Platform:          Testing 1 Action:            Raised 'sort_buffer_size' by 25%; reran marketing's problem 'ORDER BY'  Special needs:     None Test dependencies: 201 Team members:      Tom Reagan Test status:       Complete Test results:      No impact on problem query 

Although this test did not improve the performance problem, during the course of your testing, you learned that the query sorted the result set by a column that had no index. This led to a new test:

 Test #:            363 Date:              16-October-2005 Start time:        13:18 Finish time:       13:21 Platform:          Testing 1 Action:            Added index to 'secondary_region'column in 'marketing_                    programs'table  Special needs:     None Test dependencies: None Team members:      Tom Reagan Test status:       Complete Test results:      Results came back in 11 seconds vs. 65 seconds. See test 204