Training Evaluation: The Best Ways to Assess Success

Use training evaluation models to ensure success in every workforce training development initiative you undertake

If I had a hammer

You know how somebody with a hammer sees everything as a nail? After a dozen years measuring and assessing the impact of workforce training, I now have a hard time not seeing things from the perspective of the training evaluation models I use in my day-to-day work. When a situation arises that involves some sort of significant change or process improvement, I react like Pavlov’s dogs – without the saliva, mind you. Evaluation constructs whirl through my head, framing my view on how ultimate value – i.e. business results – can be shown for those initiatives.

What hammers do I use? The principal ones are the Kirkpatrick Model and the Phillips ROI Model, espoused respectively by Jim and Wendy Kirkpatrick of Kirkpatrick Partners, and Jack and Patti Phillips of the ROI Institute. Both frameworks start with similar levels of evaluation. Here is a brief overview of those first four levels, as described in the Kirkpatrick Partners website:

Level 1: Reaction

To what degree participants react favorably to the training.

Level 2: Learning

To what degree participants acquire the intended knowledge, skills, attitudes, confidence and commitment based on their participation in a training event.

Level 3: Behavior [or Application or Transfer]

To what degree participants apply what they learned during training when they are back on the job.

Level 4: Results [or Impact]

To what degree targeted outcomes occur as a result of the training event and subsequent reinforcement.

The Ultimate Value

In terms of showing ultimate value, the ROI Institute also focuses on a Level 5, Return on Investment (ROI). In turn, the Kirkpatricks speak of Return on Expectations (ROE). ROI is the well-known financial calculation showing the net benefit of an initiative, expressed in dollars. ROE (not to be confused with the financial term “return on equity”) is a non-financial indicator of ultimate value. Dr. Cathy Stawarski, director of training evaluation services for my company, PTG International, describes the difference between ROI and ROE here.

Most workforce development initiatives involve some amount of each of these five levels. Usually there’s a learning component, formal or informal. As the initiative sponsor or program support staff, you hope that the learning is viewed as positive and helpful by those receiving it (Level 1), and of course you hope they actually learn what they are intended to learn (Level 2). Then you want them to apply the new information (Level 3). Finally, you aspire to achieve impact (Level 4), for which – if you’ve planned properly, and collected the necessary benefit and cost data – you can actually calculate an ROI, or speak to an ROE (Level 5).

The ROI on Tooth Brushing???

I can regale you with ways I have helped my clients apply these training evaluation models to their workforce development efforts. But let’s talk about something down-to-earth: brushing teeth! Yes, in rearing our three boys (currently ages 18, 13, and 9), I actually spend time thinking about how to assess this twice-daily (and sometimes torturous) routine with my evaluation hammer. Here’s what it might look like:

Level 1: Reaction

Like other parents, my wife and I set out to teach our sons proper dental hygiene. In the early days, we made it fun and even sang a special teeth-brushing song. They enjoyed honing a “big-boy” skill.

Level 2: Learning

Then a dentist would assess how well the boys learned to brush their teeth and provide follow-on consultation to ensure learning had, er, taken root.

Level 3: Behavior [or Application or Transfer]

Now will the boys brush their teeth on their own? This level is arguably the number one cause of parental teeth grinding. You know how it is: “But I don’t wanna brush my teeth!” or “Why do I have to?” Not that I’m comparing adult workers to whiny children, but often the repeated behavior needed at Level 3 is where ultimate impact is achieved or not.

Level 4: Results [or Impact]

Back at the dentist office, we hope the “c” word is not pronounced. Number of cavities is a clear metric for evidence of “success” or “failure” of our teeth brushing training. Do the kids need remedial training? Or can we boast about their pristine teeth?

Level 5: ROI/ROE

As my boys get older, I spend more time talking about the “return” on their efforts: “Would you rather invest a few minutes a day now, or thousands of dollars of your own money in the future dealing with your tooth decay?” They don’t always grasp the potential negative consequences, but that’s a lot of money for video games, so at least it it gets them thinking.

Of course, I must mention the snow

Brushing teeth is a fairly mundane analogy for how to look at the world through evaluation lenses. I could give you more examples – like why shoveling 3 feet of snow from my driveway after record-setting snowstorm Jonas makes me think of evaluation models, but then you’ll just think I’m geeky for calculating that my healthy-toothed family moved 14 tons of snow this past weekend.

Do you have standard “models” that you constantly turn to in your life to help you frame your experiences? Have you applied the Kirkpatrick or Phillips training evaluation frameworks in your own professional or personal life? If so, I’d love to hear about them in your comments.

Until next week – keep those teeth brushed and let me know if I can help you apply training evaluation models to your non-mundane workforce development efforts.

Eugene de Ribeaux is part of the GovLoop Featured Blogger program, where we feature blog posts by government voices from all across the country (and world!). To see more Featured Blogger posts, click here.