The Great KPI Show Down
What is a KPI?
Key Performance Indicators (KPIs) have been used by businesses to measure whether an organisation, team, or employee is meeting a predefined goal. It is related to, but different from, metrics which display the performance of tasks, processes, or activities. To be effective, KPI should also follow the SMART (Specific, Measurable, Achievable, Relevant and Time-bound) framework to ensure that goals are realistic and can be met. For more information see KPI basics and KPI definition and examples
Peter Drucker
If you can’t measure it, you can’t manage it.
Peter Drucker, often heralded as “the man who invented management”, stated that you need to quantify and observe something in order to understand and improve it [source].
This is something at the core of the SRE/DevOps paradigm: if it moves, graph it. It is something that any developer knows when looking at optimising their code. First, measure. Second, try something. Finally, compare the before and after.
KPIs have been used to look at individuals, teams, and organisations to look for quantifiable measures of progress towards a desired result. The desired result is generally set by leadership to improve some aspect of the business. A good set of KPIs will help steer the organisation to the right direction. It takes the guess work out of the process. It gives a clear direction to those teams and individuals as to what matters most.
This can only be a good thing, right?
Right?
…
Goodhart’s law
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
Charles Goodhart stated this in 1975. Of course, Goodhart’s law [source] is not the first time this has been said and is a perfect example of a law of unintended consequences.
Humans, as a species, are extremely adaptative at gaming any system. This is why we do so well as a species. If we are under some pressure to do a thing to get a reward, we will ensure that we do all that can be done in the most effective way to ensure we meet those targets. An example of this are murder closing rates in the Police: it is quicker to pin the murder on the first person that fits than actually investigate them.
In the software world, a company which shall remain nameless instituted the following KPI: For developers, bug fixes per week. For SQA staff, bugs found per week. It was amazing! There were many bugs found and fixed. Management was happy and everyone got a bonus for smashing the targets! That is until the day where a list of bugs created by the developers was intercepted before it reached the SQA team… Simple: developer write a bug, tells SQA where to find it, and both get rewarded from the KPI.
When a measure becomes a target, it ceases to be a good measure.
So, are KPIs worthless after all?
Harmony or Discord?
When a measure becomes a target, it ceases to be a good measure.
It depends. A good KPI can be a great measure of progress, but it can also be obfuscating what is actually happening. A KPI needs to follow the SMART framework to ensure it has a chance of falling into the former category. It is a tool that, as a hammer, can be used to great effect. If used wrong, then it can be a tool for discord.
Imagine, if you will, a day when all the drama is removed from your software production: no panic, no crisis, just smooth software releases that exceed your customer’s expectations. This is what we have done in the past and can do for you.