Before rolling out any hardware or software, our team tested it in a controlled testing facility. We planned to build a computer room with several computers that would act as a model for the Rockwell Collins environment. It would replicate our technology, browsers, and computer configurations so that our team could troubleshoot any course or software before end users had access to it.
We also planned to test our training in a controlled lab to verify its effectiveness, without involving the confusion of outside influences. While it's challenging to isolate training's impact, clear, unpolluted success statistics are a powerful tool for winning the hearts and minds of leadership. We've found approaching measurement from the front end makes it possible to isolate a single successful program as an example of your overall efforts. One set of measurements can be all you need.
Early on in Butler's career, he discovered the power of a single batch of unadulterated training statistics. While developing training for the sales team at a mortgage loan company that was struggling with the efficacy of its training program, he suggested conducting a year-long comparison study. The company divided the 400 salespeople in half, making sure each group included all levels of performers. Half of the team went through the old training program and half went through Butler's new program. Then they tracked their productivity. Butler found substantial increases in the second team's performance, increases that continued over the five years the team was measured. He also found that the middle performers showed the most significant improvements after completing the training. Based on those statistics, senior management agreed to overhaul the entire training program for the company.
At most companies, the value of training has never been substantiated to this degree because measurement is approached after the fact, when outside influences make it impossible to verify training's impact. Approaching measurement in the beginning, however, makes it possible to compile the level of compelling results that Butler found at the mortgage company.
Using that model, we planned to do measurement from the front end of a single set of courses and we were able to create a scenario that directly compared the old way with the new and produced hard results.
We planned to produce a new computer-based version of a classroom course, put test groups through both formats, then test and compare each group's ability to perform the related tasks .
The question was which courses to measure. Initially it seemed prudent to measure the high-end custom courses. They require the most investment, so they should produce the most value. However, proof that your high-end custom courses work shows only that your best stuff is valid. We measured our lowest common denominators, the courses that follow the loosest standards and take the least amount of money and time to produce. If you prove that your cheapest, least complex courses have value, it becomes reasonable to assume that every course you invest in that follows your established set of guidelines and development processes will be even more effective.