A Little Background on Evaluation


One of the most widely used evaluation models was originally described by Donald Kirkpatrick (1994) in the 1950s and updated in the mid-1990s. It was extended to include return-on-investment (Phillips, 1997). Kirkpatrick defined four levels of evaluation as

  • Level 1 ‚ reaction

  • Level 2 ‚ learning

  • Level 3 ‚ behavior

  • Level 4 ‚ results.

Phillips ‚ s fifth level is known as level 5 ‚ ROI. Kirkpatrick ‚ s method was originally developed for training programs. Together, Kirkpatrick ‚ s and Phillips ‚ s methods can be applied to a broader range of WLP interventions. See table 10-1 for a brief explanation.

Table 10-1. Levels of evaluation (Kirkpatrick, 1994; Phillips, 1997).

Level

Definition

1. Reaction

Determines if the participants in a program liked the program. Level 1 evaluation forms are commonly referred to as smile sheets. This level of evaluation is collected immediately following an intervention and is the most commonly used form of evaluation. The belief is that if participants reacted well to the training or other intervention, they are more likely to implement it. If they did not like it, they are less likely to apply it back on the job.

2. Learning

Determines if the participants learned the information, skills, or behaviors presented in the intervention correctly. Participants may have liked their training or other intervention, but if they learned incorrectly, then their application of what they learned will not have the desired impact on the job.

3. Behavior

Determines if the participants implemented some or all of the desired behaviors back on the job. The participants may have liked the intervention and learned well, but if they don ‚ t do what they were supposed to do, there is no benefit to the organization.

4. Results

Determines if there was a benefit to the organization in the participant ‚ s application of new knowledge, skills, or behaviors. It is possible for participants to apply what they learned but to have no change or even a negative change in financial impact to the organization. One way that results can be thought of is as tracking gross benefits to the organization. Gross benefits means that the costs of the intervention have not yet been accounted for. It is important to determine if the changes from the learning or performance intervention were worth more than the cost to implement the intervention.

5. ROI

Phillips introduced a fifth level of evaluation. That level is ROI evaluation. A simple way of looking at ROI evaluation is to determine the net benefits to the organization or, in other words, to account for the costs of implementing the intervention versus the isolated gross benefits gained . A gross benefit less its costs is said to be a net benefit. (See chapter 9 for method of calculation.)

Levels of Evaluation

The levels of evaluation described in table 10-1 are not the only possible types of evaluation. Not everyone in the worldwide WLP community agrees that all of these levels are, in fact, forms of evaluation. Nor does everyone agree that presenting these types of evaluation in a hierarchy of levels is appropriate.

These differing points of view are often very useful as WLP professionals search for the most effective ways to study the impact of their interventions. Unfortunately, these very conversations can distract attention from an important realization when communicating value. In the Twin Cities study cited in chapter 1, when Senior executives listed the competencies they wanted from human asset professionals, knowledge of WLP or, in this case, determining what is or isn ‚ t correct about different models of evaluation was not even mentioned. Knowledge of the basic elements of the WLP profession itself was so taken for granted that it didn ‚ t even make the list of critical skills executives were looking for in WLP professionals.

Because knowing the craft is a given and because audiences often look at WLP professionals as if they just didn ‚ t get the code, it ‚ s time to step back to reflect. Rather than trying to determine if a particular model of evaluation is or isn ‚ t really evaluation, let ‚ s look at what types of data your different audiences need so that you will be able to successfully communicate your value.




Quick Show Me Your Value
Quick! Show Me Your Value
ISBN: 1562863657
EAN: 2147483647
Year: 2004
Pages: 157

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net