aes18 International Evaluation Conference

Francesca Demetriou (Lirata Consulting)

Donald Kirkpatrick published the four-level model for evaluating training programs in his 1994 book Evaluating Training Programs: The Four Levels. His objective was to "provide a simple, practical four-level approach for evaluating training programs" (Kirkpatrick, 2006). Since then, the framework has been applied extensively in evaluating training and development programs.

What is clear in choosing to utilise this framework, is that the evaluation takes on a specific values frame: that these four levels (Reaction, Learning, Behaviour, Results) are the appropriate criteria in which to judge the program on. What is not clear is the relative importance of each of the four levels, and how evaluators should therefore approach synthesising findings about the four-levels to reach an overall judgement about the performance of a program.

Synthesis is an important step in evaluation, but there is a tendency for this step to occur in ways which are not rigorous, nor explicitly justified, leading to hidden assumptions behind conclusions provided in evaluations.

In the context of the Kirkpatrick model, for a training program to be considered "good", how well does each of the four levels need to perform to determine that a program is adequate, good, or excellent? Can a training program where participants learn a lot, but the results for the organisation are limited be considered "good"?

Using Jane Davidson's (2005) guidance on evaluation synthesis, and a literature review on the use and critiques of the Kirkpatrick model, this paper considers the assumptions behind the model to provide guidance for determining the relative importance of the four levels.
An example in practice is presented, where a synthesis methodology is developed for a Tasmanian community leadership program evaluation that uses the Kirkpatrick model.