When a measure becomes a target

5 results back to index

Calling Bullshit: The Art of Scepticism in a Data-Driven World by Jevin D. West, Carl T. Bergstrom

airport security, algorithmic bias, Amazon Mechanical Turk, Andrew Wiles, bitcoin, cloud computing, computer vision, correlation coefficient, correlation does not imply causation, crowdsourcing, cryptocurrency, delayed gratification, disinformation, Dmitri Mendeleev, Donald Trump, Elon Musk, epigenetics, Estimating the Reproducibility of Psychological Science, experimental economics, invention of the printing press, John Markoff, longitudinal study, Lyft, meta-analysis, new economy, p-value, Pluto: dwarf planet, publication bias, RAND corporation, randomized controlled trial, replication crisis, ride hailing / ride sharing, Ronald Reagan, selection bias, self-driving car, Silicon Valley, Silicon Valley startup, social graph, Socratic dialogue, Stanford marshmallow experiment, statistical model, stem cell, superintelligent machines, the scientific method, theory of mind, Tim Cook: Apple, twin studies, Uber and Lyft, Uber for X, uber lyft, When a measure becomes a target

The rankings ended up being influenced as much by admissions departments’ willingness to chase metrics as they did by the quality of schools’ applicants. This problem is canonized in a principle known as Goodhart’s law. While Goodhart’s original formulation is a bit opaque,*8 anthropologist Marilyn Strathern rephrased it clearly and concisely: When a measure becomes a target, it ceases to be a good measure. In other words, if sufficient rewards are attached to some measure, people will find ways to increase their scores one way or another, and in doing so will undercut the value of the measure for assessing what it was originally designed to assess.

So if you sample instructors at random, you are likely to observe a large class or a small class in proportion to the frequency of such classes on campus. In our example above, there are more teachers teaching small classes. But large classes have many students and small classes have few, so if you sample students at random, students are more likely to be in large classes.*5 Recall from chapter 5 Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure.” Class sizes provide an example. Every autumn, college and university administrators wait anxiously to learn their position in the U.S. News & World Report university rankings. A higher ranking improves the reputation of a school, which in turn draws applications from top students, increases alumni donations, and ultimately boosts revenue and reputation alike.

Significant results are strongly overrepresented in the literature, and nonsignificant results are underrepresented. The data from experiments that generated nonsignificant results end up in scientists’ file cabinets (or file systems, these days). This is what is sometimes called the file drawer effect. Remember Goodhart’s law? “When a measure becomes a target, it ceases to be a good measure.” In a sense this is what has happened with p-values. Because a p-value lower than 0.05 has become essential for publication, p-values no longer serve as a good measure of statistical support. If scientific papers were published irrespective of p-values, these values would remain useful measures of the degree of statistical support for rejecting a null hypothesis.

pages: 301 words: 78,638

Atomic Habits: An Easy & Proven Way to Build Good Habits & Break Bad Ones by James Clear

"side hustle", Atul Gawande, Cal Newport, Checklist Manifesto, choice architecture, clean water, cognitive dissonance, delayed gratification, deliberate practice, en.wikipedia.org, financial independence, invisible hand, Lao Tzu, late fees, meta-analysis, microaggression, Paul Graham, randomized controlled trial, ride hailing / ride sharing, Sam Altman, Saturday Night Live, survivorship bias, Walter Mischel, When a measure becomes a target

We teach for standardized tests instead of emphasizing learning, curiosity, and critical thinking. In short, we optimize for what we measure. When we choose the wrong measurement, we get the wrong behavior. This is sometimes referred to as Goodhart’s Law. Named after the economist Charles Goodhart, the principle states, “When a measure becomes a target, it ceases to be a good measure.” Measurement is only useful when it guides you and adds context to a larger picture, not when it consumes you. Each number is simply one piece of feedback in the overall system. In our data-driven world, we tend to overvalue numbers and undervalue anything ephemeral, soft, and difficult to quantify.

Missing twice is the start of a new habit.” I swear I read this line somewhere or perhaps paraphrased it from something similar, but despite my best efforts all of my searches for a source are coming up empty. Maybe I came up with it, but my best guess is it belongs to an unidentified genius instead. “When a measure becomes a target”: This definition of Goodhart’s Law was actually formulated by the British anthropologist Marilyn Strathern. “‘Improving Ratings’: Audit in the British University System,” European Review 5 (1997): 305–321, https://www.cambridge.org/core/journals/european-review/article/improving-ratings-audit-in-the-british-university-system/FC2EE640C0C44E3DB87C29FB666E9AAB.

pages: 339 words: 105,938

The Skeptical Economist: Revealing the Ethics Inside Economics by Jonathan Aldred

airport security, Berlin Wall, carbon footprint, citizen journalism, clean water, cognitive dissonance, congestion charging, correlation does not imply causation, Diane Coyle, endogenous growth, experimental subject, Fall of the Berlin Wall, first-past-the-post, framing effect, greed is good, happiness index / gross national happiness, hedonic treadmill, Intergovernmental Panel on Climate Change (IPCC), invisible hand, job satisfaction, John Maynard Keynes: Economic Possibilities for our Grandchildren, labour market flexibility, laissez-faire capitalism, libertarian paternalism, longitudinal study, new economy, Pareto efficiency, pension reform, positional goods, Ralph Waldo Emerson, RAND corporation, risk tolerance, school choice, spectrum auction, Thomas Bayes, trade liberalization, ultimatum game, When a measure becomes a target

Any government adopting an explicit, overarching policy of maximizing perceived happiness must confront its relationship with democratic politics. As I have argued, the Greatest Happiness principle cannot stand above politics. Another problem is what economists call Goodhart’s Law.9 A succinct definition is: ‘when a measure becomes a target, it ceases to be a good measure’. So once governments target aggregate measures of self-reported happiness, these measures cease to track ‘true’ happiness. Goodhart’s Law can be thought of as the application to human society of Heisenberg’s Uncertainty Principle in quantum physics. Put simply, measuring a system generally disturbs it.60 In human society, an important reason is that people manipulate data once it matters to them.

There is an optimistic view that these kinds of problems can be avoided by carefully chosen targets and incentive structures, but this view is not supported by most independent reviews of the audit culture.23 On the contrary, Goodhart’s Law (introduced in Chapter 5) is frequently mentioned: when a measure becomes a target, it ceases to be a good measure. Goodhart’s Law suggests that distortions and unintended consequences are an unavoidable part of any target regime. Once people or their activities are being targeted, their behaviour inevitably changes, either unintentionally or because they actively seek to manipulate the measurements.

pages: 184 words: 46,395

The Choice Factory: 25 Behavioural Biases That Influence What We Buy by Richard Shotton

active measures, call centre, cashless society, cognitive dissonance, Daniel Kahneman / Amos Tversky, David Brooks, Estimating the Reproducibility of Psychological Science, Firefox, framing effect, fundamental attribution error, Google Chrome, Kickstarter, loss aversion, nudge unit, placebo effect, price anchoring, principal–agent problem, Ralph Waldo Emerson, replication crisis, Richard Feynman, Richard Thaler, Robert Shiller, Robert Shiller, Rory Sutherland, Veblen good, When a measure becomes a target, World Values Survey

From your perspective this was a rational decision, but it’s counter to your employer’s intention. They created the bonus system to boost income, but in this case it has reduced it. This poorly set target, which led to unintended consequences, is an example of Goodhart’s Law. This states: When a measure becomes a target, it ceases to be a good measure. One infamous example of unintended consequences is from Hanoi, Vietnam, during the spring of 1902. When faced with an outbreak of bubonic plague, the French colonialists offered a small fee for every rat’s tail handed in. The tactic seemed successful at first – tails began to pour in.

pages: 249 words: 77,342

The Behavioral Investor by Daniel Crosby

affirmative action, Asian financial crisis, asset allocation, availability heuristic, backtesting, bank run, Black Swan, buy and hold, cognitive dissonance, colonial rule, compound rate of return, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, disinformation, diversification, diversified portfolio, Donald Trump, endowment effect, feminist movement, Flash crash, haute cuisine, hedonic treadmill, housing crisis, IKEA effect, impact investing, impulse control, index fund, Isaac Newton, job automation, longitudinal study, loss aversion, market bubble, market fundamentalism, mental accounting, meta-analysis, Milgram experiment, moral panic, Murray Gell-Mann, Nate Silver, neurotypical, passive investing, pattern recognition, Ponzi scheme, prediction markets, random walk, Richard Feynman, Richard Thaler, risk tolerance, Robert Shiller, Robert Shiller, science of happiness, Shai Danziger, short selling, South Sea Bubble, Stanford prison experiment, Stephen Hawking, Steve Jobs, stocks for the long run, sunk-cost fallacy, Thales of Miletus, The Signal and the Noise by Nate Silver, Tragedy of the Commons, tulip mania, Vanguard fund, When a measure becomes a target

A similar incident was observed in India during the time of British colonial rule. A reward was set for every dead cobra and so enterprising Indians began to – you guessed it – raise cobras on snake farms. The Cobra Effect is now shorthand for what is officially referred to as Campbell’s’ Law, which states, “when a measure becomes a target, it ceases to be a good measure.” Campbell says of the tendency for measurement to corrupt efficacy, “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”

pages: 428 words: 103,544

The Data Detective: Ten Easy Rules to Make Sense of Statistics by Tim Harford

access to a mobile phone, Ada Lovelace, affirmative action, algorithmic bias, Automated Insights, banking crisis, basic income, Black Swan, Bretton Woods, British Empire, business cycle, Capital in the Twenty-First Century by Thomas Piketty, Cass Sunstein, clean water, collapse of Lehman Brothers, coronavirus, correlation does not imply causation, Covid-19, COVID-19, cuban missile crisis, Daniel Kahneman / Amos Tversky, David Attenborough, Diane Coyle, disinformation, Donald Trump, Estimating the Reproducibility of Psychological Science, experimental subject, financial innovation, Florence Nightingale: pie chart, Gini coefficient, Hans Rosling, income inequality, Isaac Newton, job automation, Kickstarter, life extension, meta-analysis, microcredit, Milgram experiment, moral panic, Netflix Prize, Paul Samuelson, publication bias, publish or perish, random walk, randomized controlled trial, recommendation engine, replication crisis, Richard Feynman, Richard Thaler, rolodex, Ronald Reagan, selection bias, sentiment analysis, Silicon Valley, sorting algorithm, statistical model, stem cell, Stephen Hawking, Steve Bannon, Steven Pinker, survivorship bias, universal basic income, When a measure becomes a target

Social scientists have long understood that statistical metrics are at their most pernicious when they are being used to control the world, rather than try to understand it. Economists tend to cite their colleague Charles Goodhart, who wrote in 1975: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”12 (Or, more pithily: “When a measure becomes a target, it ceases to be a good measure.”) Psychologists turn to Donald T. Campbell, who around the same time explained: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”13 Goodhart and Campbell were onto the same basic problem: a statistical metric may be a pretty decent proxy for something that really matters, but it is almost always a proxy rather than the real thing.

Super Thinking: The Big Book of Mental Models by Gabriel Weinberg, Lauren McCann

affirmative action, Affordable Care Act / Obamacare, Airbnb, Albert Einstein, anti-pattern, Anton Chekhov, autonomous vehicles, bank run, barriers to entry, Bayesian statistics, Bernie Madoff, Bernie Sanders, Black Swan, Broken windows theory, business process, butterfly effect, Cal Newport, Clayton Christensen, cognitive dissonance, commoditize, correlation does not imply causation, crowdsourcing, Daniel Kahneman / Amos Tversky, David Attenborough, delayed gratification, deliberate practice, discounted cash flows, disruptive innovation, Donald Trump, Douglas Hofstadter, Edward Lorenz: Chaos theory, Edward Snowden, effective altruism, Elon Musk, en.wikipedia.org, experimental subject, fear of failure, feminist movement, Filter Bubble, framing effect, friendly fire, fundamental attribution error, Gödel, Escher, Bach, hindsight bias, housing crisis, Ignaz Semmelweis: hand washing, illegal immigration, income inequality, information asymmetry, Isaac Newton, Jeff Bezos, John Nash: game theory, lateral thinking, loss aversion, Louis Pasteur, Lyft, mail merge, Mark Zuckerberg, meta-analysis, Metcalfe’s law, Milgram experiment, minimum viable product, moral hazard, mutually assured destruction, Nash equilibrium, Network effects, nuclear winter, offshore financial centre, p-value, Parkinson's law, Paul Graham, peak oil, Peter Thiel, phenotype, Pierre-Simon Laplace, placebo effect, Potemkin village, prediction markets, premature optimization, price anchoring, principal–agent problem, publication bias, recommendation engine, remote working, replication crisis, Richard Feynman, Richard Feynman: Challenger O-ring, Richard Thaler, ride hailing / ride sharing, Robert Metcalfe, Ronald Coase, Ronald Reagan, school choice, Schrödinger's Cat, selection bias, Shai Danziger, side project, Silicon Valley, Silicon Valley startup, speech recognition, statistical model, Steve Jobs, Steve Wozniak, Steven Pinker, sunk-cost fallacy, survivorship bias, The future is already here, The Present Situation in Quantum Mechanics, the scientific method, The Wisdom of Crowds, Thomas Kuhn: the structure of scientific revolutions, Tragedy of the Commons, transaction costs, uber lyft, ultimatum game, uranium enrichment, urban planning, Vilfredo Pareto, When a measure becomes a target, wikimedia commons

If the output of nails was determined by their number, factories produced huge numbers of pinlike nails; if by weight, smaller numbers of very heavy nails. The satiric magazine Krokodil once ran a cartoon of a factory manager proudly displaying his record output, a single gigantic nail suspended from a crane. Goodhart’s law summarizes the issue: When a measure becomes a target, it ceases to be a good measure. This more common phrasing is from Cambridge anthropologist Marilyn Strathern in her 1997 paper “‘Improving Ratings’: Audit in the British University System.” However, the “law” is named after English economist Charles Goodhart, whose original formulation in a conference paper presented at the Reserve Bank of Australia in 1975 stated: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”

pages: 410 words: 119,823

Radical Technologies: The Design of Everyday Life by Adam Greenfield

3D printing, Airbnb, algorithmic bias, augmented reality, autonomous vehicles, bank run, barriers to entry, basic income, bitcoin, blockchain, business intelligence, business process, call centre, cellular automata, centralized clearinghouse, centre right, Chuck Templeton: OpenTable:, cloud computing, collective bargaining, combinatorial explosion, Computer Numeric Control, computer vision, Conway's Game of Life, cryptocurrency, David Graeber, dematerialisation, digital map, disruptive innovation, distributed ledger, drone strike, Elon Musk, Ethereum, ethereum blockchain, facts on the ground, fiat currency, global supply chain, global village, Google Glasses, Ian Bogost, IBM and the Holocaust, industrial robot, informal economy, information retrieval, Internet of things, James Watt: steam engine, Jane Jacobs, Jeff Bezos, job automation, John Conway, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John von Neumann, joint-stock company, Kevin Kelly, Kickstarter, late capitalism, license plate recognition, lifelogging, M-Pesa, Mark Zuckerberg, means of production, megacity, megastructure, minimum viable product, money: store of value / unit of account / medium of exchange, natural language processing, Network effects, New Urbanism, Occupy movement, Oculus Rift, Pareto efficiency, pattern recognition, Pearl River Delta, performance metric, Peter Eisenman, Peter Thiel, planetary scale, Ponzi scheme, post scarcity, post-work, RAND corporation, recommendation engine, RFID, rolodex, Satoshi Nakamoto, self-driving car, sentiment analysis, shareholder value, sharing economy, Shenzhen special economic zone , Silicon Valley, smart cities, smart contracts, social intelligence, sorting algorithm, special economic zone, speech recognition, stakhanovite, statistical model, stem cell, technoutopianism, Tesla Model S, the built environment, The Death and Life of Great American Cities, The Future of Employment, transaction costs, Uber for X, undersea cable, universal basic income, urban planning, urban sprawl, When a measure becomes a target, Whole Earth Review, WikiLeaks, women in the workforce

They think that it gives them a competitive advantage, and they don’t want rivals nullifying that advantage by copying it. That part is straightforward enough. But the second reason is that, like all such metrics, these stats can be juked: Branch’s algorithm is subject to Goodhart’s Law, the principle that “when a measure becomes a target, it ceases to be useful as a measure.”66 In other words, they believe that if it became more widely known just how their algorithm arrived at its determinations, it would be easier for unreliable people to act in ways that would fool it into classifying them as trustworthy. On the surface, then, this is the same reason that Google holds the precise composition of its search algorithm closely: to prevent it from being gamed by interested parties.