When we have a complex goal that cannot be measured, we often find a measurement that guides us to achieve our goal.
For example, “Becoming healthy” is a vague and complex goal, so we seek something measurable. We decide to measure our body weight. But sooner or later, the measurement takes over as the real goal. We tend to be more focused on reaching the targeted weight than becoming healthy. We will make unhealthy decisions to reach the targeted weight. By focusing too much on the measurement, we lose sight of the real goal of healthiness. In fact, we are possibly becoming less healthy, by trying to lose or gain weight faster.
The effect of quantitative measurements on decision-making processes was described by an American psychologist and social scientist Donald T. Campbell in his paper titled “Assessing the Impact of Planned Social Change,” this way: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”This is known as Campbell’s law. In simple words, this law says that when we focus on a simple measurement to guide a complex goal, that measurement becomes the goal, and the measurement starts to work against the real goal. Complex social problems always have a pseudo-simple solution that gets in the way of real issues.
In his paper, Campbell used the crime rate as an example. He pointed out that a decrease in a city’s crime rate may not necessarily be due to a true reduction in the number of crimes that have been committed but may simply reflect how the police force has changed procedures to lower the number. They may have decided, for example, to change which police encounters need to be formally recorded. They may also have downgraded some crimes to less serious classifications.
Campbell’s Law is often cited as a warning about the dangers of making data-driven decisions based on a single key performance indicator. For example, a sales manager who requests monthly reports on the number of calls each sales representative makes may unconsciously cause problems for himself if he holds this metric up as an important criterion for bonuses. According to Campbell’s Law, once the salespeople know they are being evaluated on this particular metric, they may put more effort toward making sales calls and spend less time on other important tasks, such as trying to close sales.
What Campbell’s Law describes is a known phenomenon. We know many social policies that went south because the measurement was wrong. This degradation is rarely intentional. Imagine a group of people that have a goal. To focus, they write down a metric and set up the tooling to measure it. But over time, it’s the measure and not the original goal that keeps appearing in reports and discussions. Revisiting the goal often isn’t done and people slowly start forgetting about the true end goal. They hyper-focus on the metric alone and corruption appears. The effect is accelerated even more if the original people that set the goal change. This happens easily in businesses with new hires coming in and senior people moving to different endeavours. This can happen to any group.
Measuring progress using both quantitative and qualitative indicators is essential; when using quantitative data for evaluation, the indicators can become distorted or manipulated. Campbell never argued that we should abandon all hope of using these data to improve programs and systems. Indeed, in his conclusion, Campbell emphasizes that measurement could be implemented in better ways.
So, what steps can be taken to prevent the measurement from becoming the goal? Like all things, it’s not quite that simple.
We should keep reminding everyone involved in the project about the original goal, review, and possibly change the metrics every few months. Changing metrics that are too old will force you to regularly re-evaluate whether what you measure still helps you achieve the results you’re aiming for.
We should identify ways that can uncover the distortion, corruption, or misuse of data. Multiple measures- both quantitative and qualitative should be used so that numerical data doesn’t become the only way to judge a program’s effectiveness or impact.
Have you ever experienced Campbell’s law in your life? Now, don’t let the metrics you use to guide yourself to reach your end goal, cloud your judgment, and take you farther away from the actual goal. Always stay focused on the ultimate goal.