This is the seventh in the series of blogs that I have written on the effects of the fourth industrial revolution. In my last blog I had talked about how to develop the critical competencies of “Cognitive Flexibility” and “Change management”
The challenge which most head of Learning and development and HR heads face is in terms of measuring the effectiveness of the various developmental programs. Organizations spend a lot of time and money in designing and implementing learning and development interventions, however, it becomes difficult to measure the effect of these interventions. As most of us know, the CFO is a very difficult man to convince.
In 1959 Donald Kirkpatrick devised a method to evaluate the effectiveness of training and to date it remains the most used method of evaluating the effectiveness of learning and development interventions. The “Kirkpatrick model” as it is commonly known defines four steps to evaluate the effectiveness of training. The four steps are summarized below:
a. Level One: Evaluating Reactions: Measures how participants value the training. And if they were engaged, and whether they believe they can apply what they learned. This is commonly done through end of course feedback surveys.
b. Level Two: Evaluating Learning: Measures whether participants actually learned from the training. This is commonly done through post workshop quiz and observation of the way a participant executed a skill.
c. Level Three: Evaluating Behavior: Measures whether training had a positive effect on job performance (transfer). This is normally done through observation at work, focus groups and evaluation by peers/team/manager.
d. Level Four: Evaluating Results: Measures whether the training is achieving results. Has training resulted in better quality, increased productivity, increased sales and better customer service? Evaluation could be quantitative in terms of revenue/cost/quality parameters etc.
If we examine most of the competency development interventions in organization, we find that a majority of them only measure the participant’s reaction through end of course feedback on delivery of training and some quizzes (Level One and Two of the Kirkpatrick model). Very few organizations, go beyond these first two levels of the Kirkpatrick Model as it becomes difficult to implement the third and fourth level assessment. However, in case we want to clearly measure the effectiveness of the learning and development intervention ,it is imperative that the intervention be evaluated at third and fourth level. If that is done, it also becomes easier to convince the CEO and the CFO to loosen the purse strings for the next intervention.
I have found that the third level of evaluation is can be clearly measured through a 360 degree survey around 3 months post the developmental intervention. In most of the interventions I do, I spend some time to clearly understand the outcomes desired from the intervention by interacting with the key stakeholders in the organization. Based on this understanding, I design the intervention as well as the questions for the 360 degree survey through which the effectiveness of the intervention will be evaluated. This 360 degree feedback is sought before the start of the developmental intervention and three months post the formal close of the intervention. This gives a very valid measure of the amount of improvement displayed by the participant as observed by the relevant stakeholders.
The fourth level is relatively complex and difficult to measure as business results are a function of multiple factors. It is very difficult to clearly measure the change in business results which is due to training and which are due to other factors like business environment, competition etc. One of the things which I try is to ask the participants to pick a project which will have revenue or cost impact and which uses the competencies developed during the intervention. The participant is required to clearly define the competency being applied and how its application will effect these two parameters. The project picked needs to be approved by the participants manager so that there is an independent validation of the results of project. The project lifecycle is typically 6-9 months and the participant is required to make a presentation to the senior leadership team of the organization at the end of the project.
Do let me know what you think of this blog. Would love to hear from you as to what you have tried in terms of evaluation of learning and development interventions.
My next series of blogs will cover various aspect of leadership.
.
Rajiv is the principal consultant at R Square Consulting. This is seventh and the last in a series of blogs, which discussed the effect of the fourth industrial revolution and the competencies required to be successful in the workplace of 2020. Rajiv can be reached at rajiv@rquareconsult.in for any query, discussion or any professional requirements.
About R Square consulting : R Square consulting provides end to end services in the field of building leadership and managerial capability to include leadership development interventions based on a holistic blend of exploration, reflection, action learning and coaching, assessment centers and various Organizational development interventions.
We are a HR consulting firm providing customised and holistic HR interventions for developing the human capital of an organization through:
Executive Coaching
Assessment and Development Centers
Organizational Development Interventions
Leadership and managerial development
Flexi HR support for SME
