Goodhart's Law: When A Measure Is A Target, It's Not A Good Measure

Goodhart’s law is an adage stated as “When a measure becomes a target, it ceases to be a good measure.” Systems can be gamed. When you are rewarded for something, you’ll optimize your actions in a way to get the desired result. This doesn’t mean you contribute to the greater good however. Take car sales. If you are a salesperson on commission that is calculated by the number of car sales each month, you will aim to sell more vehicles. Even if that means the business incurs losses.

There are countless real-world instances of Goodhart’s law. The International Union for Conservation of Nature (IUCN) has a measure of extinction. It’s how environmental protections are decided on. So the IUCN just became more conservative in what’s considered extinct. In healthcare, hospitals striving to reduce length of stay (LOS, a key metric) inadvertently discharges patients early. Emergency readmissions rise but, hey, at least people get home faster. When teachers focus solely on test scores, they neglect broader education to meet those numbers. Metrics lose effectiveness when they are turned into goals.

Understanding Goodhart’s law helps you make strategic decisions. Especially when it comes to incentivizing people. You don’t want your customer care wing to aim to minimize call duration, since they’ll neglect solving issues thoroughly. You don’t want your athlete striving toward a particular metric, since they might pump themselves full of performance-enhancing drugs to hit those targets. You don’t want your company to emphasize quarterly earnings, since you’ll cut long-term investments or quality and harm future growth.

It applies both to individuals and teams. Be careful not to form perverse incentives. A narrow curriculum where students get incredible standardized test scores tells you nothing when broader learning is jeopardized. A sales team handling high volume gives you no data on what profits are genuinely being made, not mentioning long-term customer relationships. When the system can be gamed, you have to be doubly cautious on what you incentivize.

Real-life examples of Goodhart’s law:

• Education: schools and teachers are evaluated on their classrooms’ standardized test scores. Instruction becomes overly focused on test preparation, not broader education goals like critical thinking and creativity. When test scores become the target, they are no longer an accurate measure of student learning.

• Public Policy: a government sets a target for reducing crime rates. The basis is police arrest numbers. Law enforcement adheres to the goal by focusing on making more arrests for minor offenses. No root cause of crime is addressed. The metric (arrests) is manipulated and reflects nothing about public safety.

• Business: a company sets a goal to increase website traffic. To measure marketing success. Marketing efforts shift significantly toward getting clicks via sensational or misleading content. Not engaging potential customers with quality information. The result is a hodgepodge of standout crap on the site. Customers lose respect for the firm. Conversion rates drop.

• Healthcare: hospitals are incentivized based on the speed of patient discharge. In a bid to be seen as a superior hospital, they discharge patients before they have reached full recovery. This compromises patient care and safety. The discharge time becomes a terrible indicator of actual patient well-being.

How you can use Goodhart’s law as a mental model: (1) find what’s critical — determine which metrics genuinely depict performance in your context and use multiple to capture the full picture (e.g. not just website visitors, but their demographic, acquisition cost, and conversion rate); (2) don’t overemphasize — don’t make one metric the sole focus, since this’ll result in other metrics taking a downturn; (3) carrot or stick — be careful about what you incentivize, considering where variables can be manipulated or gamed (e.g. revenue by more sales via discounts but lower profit margins); (4) scan for unintended consequences — be on the lookout to detect when behaviors are distorted solely because of one metric; (5) results over numbers — base yourself and your team on the value of the broader mission and not short-term metrics.