3.7 FEEDBACK

< Day Day Up >

Of course, having the models, defining the measures, collecting the data and even analyzing that data is not the end of the story. You now have to present or feedback the information to managers. This is an art in itself and there are some classic examples around of how it should not be done which I am sure everyone with any management experience has come across.

Now this is an area where a trained statistician can be worth his or her weight in gold and much of what follows is the result of assistance given to me by such individuals over the years.

The first thing to remember is what is required from such feedback, what does your customer, a manager, want? Well, what is not required is an example of how clever you have been collecting all this data and collating it into nice tables pages long! Managers are, by definition, simple folk; otherwise they would still be programmers or technical designers. They are also busy, or believe they are, which amounts to the same thing, and as a result of this have only a limited amount of time available. Do not expect them to analyze data sets or to develop relationships between results. That, as the presenter of this information, is your job.

Remember also that different people relate to different styles of presentation, and as you may not know everybody who will see your feedback of information you will not know whether or not they relate to pictures or words. For this reason I advocate the use of pictures, in the form of graphs (which I will talk about later), together with text. Use the text to briefly summarize what the graphs are saying but also use a separate section or paragraph to emphasize any points and relationships you feel are important. Simply presenting the summarized data is seldom adequate.

I favor a management summary at the front of any report that summarizes the summaries in both words and pictures and I try to use this to point out potential problems. Do not be afraid of doing this. Provided you maintain a reasonable tone to the presentation and do not insist that your interpretation is the only one, most managers quite like to be told what is happening. It makes a change for them to get usable information, and they like to be told things.

There are some important points to note about management information that relate to feedback or presentation. Some people lay great store by the idea of Statistical Process Control. This is an extremely useful concept and approach but it does depend upon a process or product attribute being statistically stable. By this I mean that most variation is predictable. For example, assuming I was any good at archery, the process of firing an arrow at a target would probably be statistically stable in that my arrows would normally be clustered within a certain circumference away from the gold. If I then shot an arrow and it hit the target rim it would be exceptional, in statistical terms an outlier, and I should ask why it happened. I may investigate and find that the arrow was warped so I could improve the process of firing by not using that arrow again. Unfortunately, I have problems hitting the target, let alone getting a predictable cluster. Think about it—in those circumstances, statistical process control may not be appropriate.

For a software system I may also observe statistical stability when, say, reliability values in terms of defect density for a system enhanced and released quarterly, remain within a certain range for a given space of time. Once I have this stability I can set two bounds around the norm. If I am using defect density as a reliability measure, then if the value increases it is bad. Buut remember that, if statistically stable, just because the current value is worse than the previous value it is not a cause for concern (provided the current value is within the expected range). I should set a control bound at some value greater than the highest expected and acceptable value. You will find references to three or two sigma, usually defined as standard deviation from the mean, for this control bound but I believe that common sense and your own business constraints are a much better guide. This is especially true when dealing with software systems. If a value exceeds the control bound then it should trigger investigation and, potentially, corrective action.

The other bound I can set is a target for the attribute value. Using our reliability example I would set a target at some value lower than the least expected value. I assume that the target is reached when the values for reliability over some period of time, not just for one data point, or the associated trend values are lower than the target value.

Of course, norms and control bounds should be periodically reassessed. Target levels should be changed once the target has been met, but do make sure that everyone concerned knows the target has been reached and make sure that they all get a pat on the back.

< Day Day Up >