10.1 PILOT PROJECTS

< Day Day Up >

Pilot projects can occur for a number of reasons. It may be that, during the feasibility study stage, you have come across a metric-based technique that, for some reason, appears so elegant, so perfectly suited to what you know you want to do that you immediately see it forming part of your Software Metrics program. It may be that as you go through the design stage you identify another metric that you wish to test out within your own environment. Or, it may simply be that you wish to evaluate a set of tools, perhaps cost estimation packages, with a view to including one or two within the program.

For any of these reasons you may wish to run a pilot project at some point during the design stage. To be honest, pilot projects can be kicked off almost anytime although it would make sense to know what requirement the pilot is supposed to satisfy before committing time and resources.

Pilots should be small, targeted and of as short a duration as possible. You should carefully consider the experimental design of your pilot to be sure that you can get the results that you are aiming for, and of course that you are not guaranteeing the results by the design of your pilots. Experimental design is another specialty, just like statistics. My feeling is that I know enough to know I need specialist help and I try to check the design of pilots out with such a specialist unless I have done such a pilot before. If you do not know anyone within your own organization who can help, and if you have an operational research group I suggest they be your first port of call, then try the local educational facilities, colleges, polytechnics and universities. Academics can help you with this area and usually do some consultancy at very reasonable rates. One day's consulting could be money well spent.

Remember to review the results of your pilot and to feed them into your design process. Very often so called pilots are nothing less than the first stage of implementation with no feedback loop in place to affect the design. This is fine if you are just playing word games to get things rolling but you can find a genuine pilot becoming that implementation unless you specifically build that feedback loop. If you need convincing of this see if you have any development teams using rapid prototyping as a development methodology. If you have, ask them how many prototypes they throw away!

Done right, pilots can provide a wealth of useful information, especially about the amount of effort that will be involved in the full-scale implementation of an aspect of a measurement initiative, and can save considerable amounts of money. This is especially true of the pilot that appears to fail. In other words, when the results of the pilot is not what you hoped for, but is it not better to find out that something does not work in a contained, controlled experiment rather than twelve months down the line when you have invested more money than you should have?

I would like to give you two examples of pilots. In the first case, I was involved in the piloting of a specific sizing technique in a real-time environment. By analyzing historical data we were able to determine that our use of the size measure in that specific environment did not work. By that I mean that there was little correlation between size, as we were measuring it, and effort. It was much better to find this out at that point rather than to have started implementation of the sizing technique with all the training, development of storage mechanisms and the building of feedback loops that would have been involved. Such an exercise would have been costly and would have resulted in a major embarrassment if we had then found that there was no foundation for using the sizing technique.

The second pilot I will touch upon briefly involved the use of a simple cost estimation strategy. By piloting the strategy within one part of the organization, we were able to develop a familiarity with the techniques involved and, most importantly, we were able to obtain in-house examples that demonstrated the successful use of those techniques.

So, bearing in mind that pilots can be going on and feeding into the main design streams we will now start to talk about those streams.

< Day Day Up >