In this last edition of the change management series, we will cover Key Performance Indicators. Based on the adage “What gets measured gets done”, we will look at how to leverage KPIs to monitor and drive change.

To introduce change, one should focus on changing behaviors first, then only in a second stage looking at results, because the expected results from these new behaviors come gradually from learning and practice. We therefore propose to use a combination of KPIs:

1. When we introduce a new way of working, the first step is to measure if people do what they are supposed to. This can be seen as a sort of compliance to a new process.
2. Doing something wrong does not help, so we should also measure the quality what is done.
3. And finally, we should measure whether we reach the benefits expected with the change. If not, some fine-tuning the new way of working might be required.

Let’s illustrate this with the implementation of new project management standards in an IT organization.

A new project management system had been implemented and training provided. However lower attention had been given to the understanding of project management and its integration with other company processes like portfolio management and budgeting. As a result, people maintained the system based on the guidelines to cover mainly resource planning but they did not change the way they managed projects. The resource capacity planning was improved and helped preparing budgets, but performance in project management was not reached: projects experienced significant delays and went over budget. Also recurrent project portfolio management did not really take place. Projects were living a life of their own once approved and sensing real issues took a long time before questioning whether a project had to be stopped.

So we decided we needed to build strong foundations in project management before project performance could really be improved. Resource management was one of the issues but not the only one. We organized a second series of training tying project management best practices with the system and explained to the people the reasons for the guidelines in the system. And this triggered strong discussions during training about the need to have a supporting portfolio management process in parallel with project management.

We used the idea that a mix of “enabling” KPI and “result” KPI allow for effective change monitoring. And we ensured that these KPIs were monitored and acted upon by top management, otherwise it was much more difficult to use them to drive behavior change.

Three categories of KPIs were defined:

1. Process complianceExample: % Projects with properly filled in project charter or monthly status reportThis KPI shows if people do what they are supposed to do but does not tell if they do it right, hence the next family of KPIs:
2. Process qualityExample: resource allocation between 80 and 120% over the next 8 weeksEven if the process is followed the right way, it does not mean that the organization is reaping the benefits of the new methodology. It is therefore also important to focus on targeted outcomes:
3. Project PerformanceThis type of KPIs can then in turn question the methodology itself if people comply but benefits are not reached: some further fine-tuning of methodology can be necessary.

Example: variance on time, whether time to next milestone or entire project duration, variance on effort or budget. +/-10% is typically considered good and being too far below the baseline is also not good, because it means the estimates are not accurate enough.

The three categories are complementary and focus on changing behaviours to reach pre-defined benefits.  An example of actual KPIs is shown below.

Another interesting and related example comes from the book “The Project Management Scorecard” by J.J. Phillips, T.W.Bothell, G.L. Snead, Butterworh Heinemann. It suggests 5 levels of KPIs for measuring the success of project management solutions:

1. Reaction/Satisfaction and Planned Action- the satisfaction score on the solution in End of Project questionnaire captures after each project the feedback on the solution and specifies which guidelines are most useful
2. Learning- the satisfaction score in learning assessment indicates whether training participants are convinced and intend to follow the new practices covered during the training
3. Application/implementation- the frequency of use of skills via Follow-up questionnaire indicates how often the new practices are effectively used during projects
4. Business Impact- KPIs like weekly production per product line for project improving manufacturing production delays show the effective results of the project management solution
5. ROI- translates operational results into financial benefits for the company.

We can somewhat relate these levels to some of the ADKAR model stages (that we reviewed in our previous article, giving 5 main steps towards change- Awareness, Desire, Knowledge, Ability, Reinforcement; please refer to the article for more explanations):

• Reaction covers the Desire stage
• Learning covers the Knowledge stage
• Application covers the Ability stage

Moving down the levels acknowledges the fact that the changes must go through a number of steps before reaching long-term results. It can be a good idea to develop KPIs related to the ADKAR change stages. We can then use these to determine which stages to focus on to speed changes.

The 2 previous examples focused on project management.

Let’s give another example of change KPIs in the manufacturing world. The project introduced a Production Supervision System intending to decrease Production Down-Time by logging and analyzing down-time on production lines bottlenecks.

In this case, an obvious choice for the KPIs was related to the new behavior of logging down-time reason for each production plant, allowing to give an overview across plants and put peer pressure on sites to drive change. Global management wanted to widen the usage of a system to drive increased Overall Equipment Efficiency at group level by 2% over 2 years. This was the result KPI.

The system was redesigned to ease usage and increase scalability and was deployed in 32 plants worldwide. Each plant had to start using the application after the system was installed and a global training delivered. Showing the adoption rate of the new behavior increased visibility of status at management level and pushed the priority forward: initially no significant improvement was noticed in terms of usage, but at a certain moment when shown to and discussed with top management, in a few months a dramatic increase of system usage was observed (see picture below).  No plant manager wanted to be pointed at by the VP Global Manufacturing, which enabled the project to move up the list of priorities within plants and to see change in usage.

This in addition to close cooperation with local key users to train local resources after being trained themselves centrally as member of a global community ensured people felt comfortable with the new systems and that any concern could quickly be addressed.

Here too the same principle of monitoring KPIs related to new behavior was applied successfully.

Note that the usage of checklists can also help change behaviors (after training, to reinforce). For example, coming back to project management improvement, 2 kinds of checklist can be used:

1. For the PM alone: the checklist focuses actions/ practices  and answer the question :”do I follow the methodology?”
2. For the hand-overs at different times in the project, the checklist can focus on content/needed information at that time:

• Project start: hand-over from demand/portfolio manager to project manager
• Project go-live: hand-over from IT project manager to business
• Project closure- hand-over from project manager to operations/ support structure
• If applicable: Project hand-over to new project manager

The checklist formalizes agreement between the 2 parties that hand-over has been properly made to avoid disruption in project.

The advantages of using a checklist are multiple: focus on key points, easy to use and read, can be used with team

 

Monitoring the usage of checklists can be a good idea.

 

Articles on the same topic: 

7 kinds of business changes/transformations

6 main drivers of business changes