Occam’s Razor visits the 3rd Level of Kirkpatrick
Posted on September 8th, 2014

by Randy Sabourin

occumsI’m not sure I can get the title to be any more jargon-y, but I like it. Perhaps we should get definitions out of the way first. Occam’s Razor is a problem solving principle devised by William of Ockham in the 12th century. It states that among competing hypotheses, the one with the fewest assumptions should be selected. Its modern adaption, that the simplest explanation is often the most likely one, can be applied to complex problems to help simplify them. Kirkpatrick’s 4 Levels of Learning Evaluation was developed to measure the effectiveness of training by Donald Kirkpatrick. The 3rd level is the Holy Grail of Corporate Learning. It asks if there is an observable change in the participant’s behavior from the training. It’s generally evaluated 3 to 6 months after the training. I’m not going to define the Holy Grail.

Sustaining learning, and observing behavioral change in participants are the toughest challenges in Corporate Learning and Development. Performance issues such as poor customer service ratings, stalled growth in sales, lack of a coaching culture, and inadequate leadership skills indicate that the learning didn’t stick and that there has been no meaningful way to measure growth in participants. We’ve applied Occam’s Razor to the problem with the intent of not only measuring the change and sustainment of each participant or cohort, but to also assist them to improve their skills in the process. Measuring the impact of Corporate Learning often becomes an arduous journey that involves surveying internal stakeholders who have a vested interest in being perceived to use the new learning. The simplicity of our approach delivers a consistent, unbiased, third party measurement over a large cohort across a diverse geography. Razor applied.

Here’s a typical approach we use and have had success with.

1. Transferring Knowledge:

In traditional corporate learning, most corporations use e-learning, traditional workshops, and/or a combination of mediums. With this approach the participants have gained knowledge, but a limited amount of skill. Most corporations have a well defined process and effectively measure Kirkpatrick Level 1 (Reaction – what participants thought and felt about the training (satisfaction; “smile sheets”) and Level 2 (Learning – the resulting increase in knowledge and/or skills, and change in attitudes. This evaluation occurs during the training in the form of either a knowledge demonstration or test). Recognizing that knowledge often doesn’t translate into ability, more and more of our clients are including Live Roleplays and Simulations and using professional Roleplayers, as opposed to peer to peer roleplay, to help the transition from knowledge to skill.

2. Turning Knowledge into Skills

We recommend that 4 to 6 weeks after transferring the knowledge each participant participate in a scenario-based 1:1 Roleplay Practice, with a professional Roleplayer, to translate that knowledge into ability and to help increase the skill level through targeted practicing and coaching. Participants are unlikely to ‘try’ new skills in real life situations until they are comfortable with them. We enable them get comfortable through practice. During the practice, their performance is measured against the skills taught during the ‘transfer knowledge’ phase. A key component in the success of our approach is the fundamental belief that ‘safe practice’ is critical to skill development. We gather data on every skill and every participant starting with the first scenario to build baseline measurements and the foundation to effective Level 3 measurement. The data we gather tells our clients importation information; from the example below, 23% of this cohort of 750 participants could create a meaningful value statement, 14% used empathy to acknowledge concerns during objection handling, and 23% probed using a ramification question during the discovery process during the Baseline Session – Call 1. It also measures Impact Skills such as how many participants related to the client and made them feel ‘comfortable’, or how many sounded professional and confident to as to establish themselves as a ‘trusted advisor’.

Throughout this ‘practice’ phase, participants experience several situations and hone their skills in highly realistic and strategically designed scenarios. As the participant progresses through the course, we can observe an increase in the skills through the same measurement process.

3. Level 3 Measurement – The Simplicity of Occam’s Razor

As a result of the practice phase, we have a great baseline for evaluating if there has been a real change in the behavior of the participants. We engage with them again 8 to 12 weeks following their initial practice with us in another 1:1 telephone roleplay using the same skills to evaluate and measure the cohort or individual against the performance from the first scenario. The data we see below tells us that after scenario one, 23% of this cohort of 750 participants could create a meaningful value statement. By the 3rd call in the Baseline Session, after practicing with our Roleplayers, the figure was 57%. During the Measurement Call 1 (Level 3 Kirkpatrick 8 weeks later) the figure was 65%. This approach creates impressive, relevant, and real data to reinforce the value of your learning program. Additionally, we recommend that the Level 3 Practice sessions contain 3 or 4 scenarios to help participants increase their skills levels even further.

We find this methodology effective for sales or customer service, whether your team works face to face or over the telephone. It also works very well for Coaching and Leadership Development programs. Sustainment and Level 3 Kirkpatrick measurement does not need to be an elusive Grail.

razor graph

 note: this graphics compares a subset of skills from a cohort of 750 participants. Baseline Session 1 – Call 1 was conducted 3 weeks after a 1 day Curiosity in Sales Workshop. Measurement Call 1 represents the participant’s skills at week 8. 


One response to “Occam’s Razor visits the 3rd Level of Kirkpatrick”

  1. Dianne says:

    Thank you for sending this article Randy, I really enjoyed it. You guys are doing some fascinating work up there!

Leave a Reply

To learn more about us and how we can help your employees become more effective communicators through deliberate, experiential practice, please contact us.
Twitter Facebook
Toronto 416-366-6296
Greensboro 336-663-0637
Toll-Free 1-866-945-0648

550 Queen Street East, Suite 300
Toronto, Ontario, Canada
M5A 1V2