Is There a Level 5?


Is There a Level 5?

Sometimes a Level 5 is suggested, which generally involves ROI, but this is essentially a component of Level 4. Whether it constitutes Level 4 or 5 is purely academic. The chief point is getting there, not whether you call it 4 or 5.



Before You Evaluate, Preevaluate (Don't Forget Level 0)

Level 0 is what I call evaluation before any training has been done. It is a snapshot of current-state performance. You may already have done this in your front-end needs assessment, but if you didn't, and you plan on evaluating after the new initiative, then it is imperative that you establish current-state benchmarks before deploying the new solution. This may sound obvious, but astonishingly few corporations do this. In the rush to deploy new and better solutions based on technology, people overlook this first crucial step. Establish current-state baselines before you proceed. You will be richly rewarded.



The Four Levels Are Four Phases

Traditionally the first two evaluation levels are taken at the end of the class in the form of a survey of student attitudes (plus an exam). The third level is gathered on the job a month or two later, and the fourth level is generally calculated at the end of the following financial quarter. Thus the levels are actually spread out over time, and they should perhaps better be referred to as the "four phases" of evaluation.



The Origin of the Four Levels: The Domains of Learning

Although Kirkpatrick makes no reference to Benjamin Bloom's earlier work on learning objectives (see Fastpaths 1956, Bloom), Kirkpatrick essentially translates Bloom's three kinds of learning (knowledge, skills, and attitudes) into three levels of evaluation. "Attitudes" become Level 1 evaluation (opinion); "knowledge" becomes Level 2 evaluation (cognitive tests); and "skills" on-the-job become Level 3 evaluation. Kirkpatrick then adds a fourth level, namely financial impact of the training. The following chart summarizes the comparison:

Bloom's Three Learning Domains

Kirkpatrick's Four Evaluation Levels

  1. Emotional (Attitude)

  2. Mental (Knowledge)

  3. Physical (Skills)

  1. Emotional (Attitude toward the course)

  2. Mental (Tests in class)

  3. Physical (Transfer to on-the-job skills)

  4. Financial (Additional)

Fastpaths

1956

Benjamin Bloom: Taxonomy of Educational Objectives. Bloom's three domains of learning objectives are precursors of Kirkpatrick's four levels of evaluation.

1959

Donald Kirkpatrick's series of four articles in Training magazine on "Techniques for Evaluating Training Programs," in which he formulates the four levels of evaluation.

1971

James Block: Mastery Learning: Theory and Practice.

1973

Robert Mager: Measuring Instructional Intent. A classic text on testing and measurement. Republished numerous times.

1989

Dana and James Robinson: Training for Impact: How to Link Training to Business Needs and Measure the Results.

1994

Donald Kirkpatrick: Evaluating Training Programs: The Four Levels.

1995

John Noonan: Elevators: How to Move Training Up from the Basement. See his chapter on "Evaluation," with its creative suggestions for Level 1 surveys to produce Level 4 approximations in an organization.

1997

Jack Phillips: Handbook of Training Evaluation and Measurement Methods. One of several titles by Phillips, who specializes in evaluation.

1999

Odin Westgaard: Tests That Work.

1999

Richard Swanson and E. Holton: Results: How to Assess Performance, Learning, and Perceptions in Organizations.

2000

Jac Fitz-enz: The ROI of Human Capital.

2002

Judith Hale: Performance-Based Evaluation: Tools and Techniques to Measure the Impact of Training. Hale, an expert on evaluation, re-emphasizes the crucial distinction between academic knowledge-based tests (Level 2) and actual on-the-job performance-based evaluations (Level 3).

See also Return on Investment and Cost-Benefit Analysis

What gets measured, gets done.

—Roger Chevalier, 2002