publication bias

20 results back to index

pages: 402 words: 129,876

Bad Pharma: How Medicine Is Broken, and How We Can Fix It by Ben Goldacre


data acquisition, framing effect, if you build it, they will come, illegal immigration, income per capita, meta analysis, meta-analysis, placebo effect, publication bias, randomized controlled trial, Ronald Reagan, selective serotonin reuptake inhibitor (SSRI), Simon Singh, WikiLeaks

Health Technol Assess. 2010 Feb;14(8):iii, ix–xi, 1–193. 23 Dickersin K. How important is publication bias? A synthesis of available data. Aids Educ Prev 1997;9(1 SA):15–21. 24 Ioannidis J. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998;279:281–6. 25 Bardy AH. Bias in reporting clinical trials. Brit J Clin Pharmaco 1998;46:147–50. 26 Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 2008;3(8):e3081. 27 Decullier E, Lhéritier V, Chapuis F. Fate of biomedical research protocols and publication bias in France: retrospective cohort study. BMJ 2005;331:19. Decullier E, Chapuis F. Impact of funding on biomedical research: a retrospective cohort study.

Systematically taking all the evidence that we have so far, what do we see overall? It’s not ideal to lump every study of this type together in one giant spreadsheet, to produce a summary figure on publication bias, because they are all very different, in different fields, with different methods. This is a concern in many meta-analyses (though it shouldn’t be overstated: if there are lots of trials comparing one treatment against placebo, say, and they’re all using the same outcome measurement, then you might be fine just lumping them all in together). But you can reasonably put some of these studies together in groups. The most current systematic review on publication bias, from 2010, from which the examples above are taken, draws together the evidence from various fields.29 Twelve comparable studies follow up conference presentations, and taken together they find that a study with a significant finding is 1.62 times more likely to be published.

Why do negative trials disappear? In a moment we will see more clear cases of drug companies withholding data – in stories where we can identify individuals – sometimes with the assistance of regulators. When we get to these, I hope your rage might swell. But first, it’s worth taking a moment to recognise that publication bias occurs outside commercial drug development, and in completely unrelated fields of academia, where people are motivated only by reputation, and their own personal interests. In many respects, after all, publication bias is a very human process. If you’ve done a study and it didn’t have an exciting, positive result, then you might wrongly conclude that your experiment isn’t very interesting to other researchers. There’s also the issue of incentives: academics are often measured, rather unhelpfully, by crude metrics like the numbers of citations for their papers, and the number of ‘high-impact’ studies they get into glamorous well-read journals.

pages: 322 words: 107,576

Bad Science by Ben Goldacre


Asperger Syndrome, correlation does not imply causation, experimental subject, hygiene hypothesis, Ignaz Semmelweis: hand washing, John Snow's cholera map, Louis Pasteur, meta analysis, meta-analysis, offshore financial centre, p-value, placebo effect, publication bias, Richard Feynman, Richard Feynman, risk tolerance, Ronald Reagan, selection bias, selective serotonin reuptake inhibitor (SSRI), the scientific method, urban planning

If there is no publication bias, you should see a nice inverted funnel: the big, accurate trials all cluster around each other at the top of the funnel, and then as you go down the funnel, the little, inaccurate trials gradually spread out to the left and right, as they become more and more wildly inaccurate—both positively and negatively. If there is publication bias, however, the results will be skewed. The smaller, more rubbish negative trials seem to be missing, because they were ignored—nobody had anything to lose by letting these tiny, unimpressive trials sit in their bottom drawer—and so only the positive ones were published. Not only has publication bias been demonstrated in many fields of medicine, but a paper has even found evidence of publication bias in studies of publication bias. Here is the funnel plot for that paper.

If you aim too high and get a few rejections, it could be years until your paper comes out, even if you are being diligent: that’s years of people not knowing about your study. Publication bias is common, and in some fields it is more rife than in others. In 1995, only 1 per cent of all articles published in alternative medicine journals gave a negative result. The most recent figure is 5 per cent negative. This is very, very low, although to be fair, it could be worse. A review in 1998 looked at the entire canon of Chinese medical research, and found that not one single negative trial had ever been published. Not one. You can see why I use CAM as a simple teaching tool for evidence-based medicine. Generally the influence of publication bias is more subtle, and you can get a hint that publication bias exists in a field by doing something very clever called a funnel plot. This requires, only briefly, that you pay attention.

But these, at least, were transparent flaws: you only had to read the trial to see that the researchers had given a miserly dose of a painkiller; and you should always read the methods and results section of a trial to decide what its findings are, because the discussion and conclusion pages at the end are like the comment pages in a newspaper. They’re not where you get your news from. How can we explain, then, the apparent fact that industry funded trials are so often so glowing? How can all the drugs simultaneously be better than all of the others? The crucial kludge may happen after the trial is finished. Publication bias and suppressing negative results ‘Publication bias’ is a very interesting and very human phenomenon. For a number of reasons, positive trials are more likely to get published than negative ones. It’s easy enough to understand, if you put yourself in the shoes of the researcher. Firstly, when you get a negative result, it feels as if it’s all been a bit of a waste of time. It’s easy to convince yourself that you found nothing, when in fact you discovered a very useful piece of information: that the thing you were testing doesn’t work.

pages: 467 words: 116,094

I Think You'll Find It's a Bit More Complicated Than That by Ben Goldacre


call centre, conceptual framework, correlation does not imply causation, crowdsourcing, death of newspapers, Desert Island Discs,, experimental subject, Firefox, Flynn Effect, jimmy wales, John Snow's cholera map, Loebner Prize, meta analysis, meta-analysis, moral panic, placebo effect, publication bias, selection bias, selective serotonin reuptake inhibitor (SSRI), Simon Singh, statistical model, stem cell, the scientific method, Turing test, WikiLeaks

: First, Magnetise Your Wine What Is Science: BAD ACADEMIA What If Academics Were as Dumb as Quacks with Statistics What if Academics: publish a mighty torpedo: Brain-Imaging Studies Report More Positive Findings Than Their Numbers Can Support. This Is Fishy Brain-Imaging Studies: publication bias: took a different approach: ‘None of Your Damn Business’ None of Your: 2004 published a study:;78/4/1433 it was retracted: Dr L.

You’ll have seen plenty of news stories telling you that one part of the brain is bigger, or smaller, in people with a particular mental health problem, or even a specific job. These are generally based on real, published scientific research. But how reliable are the studies? One way of critiquing a piece of research is to read the academic paper itself, in detail, looking for flaws. But that might not be enough, if some sources of bias might exist outside the paper, in the wider system of science. By now you’ll be familiar with publication bias: the phenomenon whereby studies with boring negative results are less likely to get written up, and less likely to get published. Normally you can estimate this using a tool such as, say, a funnel plot. The principle behind these is simple: big, expensive landmark studies are harder to brush under the carpet, but small studies can disappear more easily. So essentially you split your studies into ‘big ones’ and ‘small ones’: if the small studies, averaged out together, give a more positive result than the big studies, then maybe some small negative studies have gone missing in action.

By working backwards and sideways from these kinds of calculations, Ioannidis was able to determine, from the sizes of effects measured, and from the numbers of people scanned, how many positive findings could plausibly have been expected, and compare that to how many were actually reported. The answer was stark: even being generous, there were twice as many positive findings as you could realistically have expected from the amount of data reported on. What could explain this? Inadequate blinding is an issue: a fair amount of judgement goes into measuring the size of a brain area on a scan, so wishful nudges can creep in. And boring old publication bias is another: maybe whole negative papers aren’t getting published. But a final, more interesting explanation is also possible. In these kinds of studies, it’s possible that many brain areas are measured to see if they’re bigger or smaller, and maybe then only the positive findings get reported within each study. There is one final line of evidence to support this. In studies of depression, for example, thirty-one studies report data on the hippocampus, six on the putamen, and seven on the prefrontal cortex.

pages: 284 words: 79,265

The Half-Life of Facts: Why Everything We Know Has an Expiration Date by Samuel Arbesman


Albert Einstein, Alfred Russel Wallace, Amazon Mechanical Turk, Andrew Wiles, bioinformatics, British Empire, Cesare Marchetti: Marchetti’s constant, Chelsea Manning, Clayton Christensen, cognitive bias, cognitive dissonance, conceptual framework, David Brooks, demographic transition, double entry bookkeeping, double helix, Galaxy Zoo, guest worker program, Gödel, Escher, Bach, Ignaz Semmelweis: hand washing, index fund, invention of movable type, Isaac Newton, John Harrison: Longitude, Kevin Kelly, life extension, Marc Andreessen, meta analysis, meta-analysis, Milgram experiment, Nicholas Carr, p-value, Paul Erdős, Pluto: dwarf planet, publication bias, randomized controlled trial, Richard Feynman, Richard Feynman, Rodney Brooks, social graph, social web, text mining, the scientific method, Thomas Kuhn: the structure of scientific revolutions, Thomas Malthus, Tyler Cowen: Great Stagnation

While some have made this out to be somewhat mysterious, that needn’t always be the case, as shown in the example of Planet X. Increasingly precise measurement allows us to often be more accurate in what we are looking for. And these improvements frequently dial the effects downward. But the decline effect is not only due to measurement. One other factor involves the dissemination of measurements, and it is known as publication bias. Publication bias is the idea that the collective scientific community and the community at large only know what has been published. If there is any sort of systematic bias in what is being published (and therefore publicly measured), then we might only be seeing some of the picture. The clearest example of this is in the world of negative results. If you recall, John Maynard Smith noted that “statistics is the science that lets you do twenty experiments a year and publish one false result in Nature.”

., 174 Godwin’s law, 105 Goldbach’s Conjecture, 112–13 Goodman, Steven, 107–8 Gould, Stephen Jay, 82 grammar: descriptive, 188–89 prescriptive, 188–89, 194 Granovetter, Mark, 76–78 Graves’ disease, 111 Great Vowel Shift, 191–93 Green, George, 105–6 growth: exponential, 10–14, 44–45, 46–47, 54–55, 57, 59, 130, 204 hyperbolic, 59 linear, 10, 11 Gumbel, Bryant, 41 Gutenberg, Johannes, 71–73, 78, 95 Hamblin, Terry, 83 Harrison, John, 102 Hawthorne effect, 55–56 helium, 104 Helmann, John, 162 Henrich, Joseph, 58 hepatitis, 28–30 hidden knowledge, 96–120 h-index, 17 Hirsch, Jorge, 17 History of the Modern Fact, A (Poovey), 200 Holmes, Sherlock, 206 homeoteleuton, 89 Hooke, Robert, 21, 94 Hull, David, 187–88 human anatomy, 23 human computation, 20 hydrogen, 151 hyperbolic growth rate, 59 idiolect, 190 impact factors, 16–17 inattentional blindness (change blindness), 177–79 India, 140–41 informational index funds, 197 information transformation, 43–44, 46 InnoCentive, 96–98, 101, 102 innovation, 204 population size and, 135–37, 202 prizes for, 102–3 simultaneous, 104–5 integrated circuits, 42, 43, 55, 203 Intel Corporation, 42 interdisciplinary research, 68–69 International Bureau of Weights and Measures, 47 Internet, 2, 40–41, 53, 198, 208, 211 Ioannidis, John, 156–61, 162 iPhone, 123 iron: magnetic properties of, 49–50 in spinach, 83–84 Ising, Ernst, 124, 125–26, 138 isotopes, 151 Jackson, John Hughlings, 30 Johnson, Steven, 119 Journal of Physical and Chemical Reference Data, 33–35 journals, 9, 12, 16–17, 32 Kahneman, Daniel, 177 Kay, Alan, 173 Kelly, Kevin, 38, 46 Kelly, Stuart, 115 Kelvin, Lord, 142–43 Kennaway, Kristian, 86 Keynes, John Maynard, 172 kidney stones, 52 kilogram, 147–48 Kiribati, 203 Kissinger, Henry, 190 Kleinberg, Jon, 92–93 knowledge and facts, 5, 54 cumulative, 56–57 erroneous, 78–95, 211–14 half-lives of, 1–8, 202 hidden, 96–120 phase transitions in, 121–39, 185 spread of, 66–95 Koh, Heebyung, 43, 45–46, 56 Kremer, Michael, 58–61 Kuhn, Thomas, 163, 186 Lambton, William, 140 land bridges, 57, 59–60 language, 188–94 French Canadians and, 193–94 grammar and, 188–89, 194 Great Vowel Shift and, 191–93 idiolect and, 190 situation-based dialect and, 190 verbs in, 189 voice onset time and, 190 Large Hadron Collider, 159 Laughlin, Gregory, 129–31 “Laws Underlying the Physics of Everyday Life Really Are Completely Understood, The” (Carroll), 36–37 Lazarus taxa, 27–28 Le Fanu, James, 23 LEGO, 184–85, 194 Lehman, Harvey, 13–14, 15 Leibniz, Gottfried, 67 Lenat, Doug, 112 Levan, Albert, 1–2 Liben-Nowell, David, 92–93 libraries, 31–32 life span, 53–54 Lincoln, Abraham, 70 linear growth, 10, 11 Linnaeus, Carl, 22, 204 Lippincott, Sara, 86 Lipson, Hod, 113 Little Science, Big Science (Price), 13 logistic curves, 44–46, 50, 116, 130, 203–4 longitude, 102 Long Now Foundation, 195 long tails: of discovery, 38 of expertise, 96, 102 of life, 38 of popularity, 103 Lou Gehrig’s disease (ALS), 98, 100–101 machine intelligence, 207 Magee, Chris, 43, 45–46, 56, 207–8 magicians, 178–79 magnetic properties of iron, 49–50 Maldives, 203 Malthus, Thomas, 59 mammal species, 22, 23, 128 extinct, 28 manuscripts, 87–91, 114–16 Marchetti, Cesare, 64 Marsh, Othniel, 80–81, 169 mathematics, 19, 51, 112–14, 124–25, 132–35 Matthew effect, 103 Mauboussin, Michael, 84 Mayor, Michel, 122 McGovern, George, 66 McIntosh, J. S., 81–82 McWhorter, John, 191 measurement, 142–70 decline effect and, 155–56, 157 kilogram in, 147–48 meter in, 143–47 of Mount Everest, 140–41 precision and accuracy in, 149–50 prefixes in, 47–48, 142, 147 publication bias and, 156 of trees, 142 Mechanical Turk, 180–82 medical knowledge, 23, 32, 51–52, 53, 122, 197, 198, 208 about cirrhosis and hepatitis, 28–30 MEDLINE, 99–100 memorization, 198 Mendel, Gregor, 106 Mendeley, 117, 118 Merton, Robert, 61, 103, 104 mesofacts, 6–7, 195, 203 meta-analysis, 107–8 cumulative, 109–10 meter, 143–47 Milgram, Stanley, 24, 167 mobile phone calls, 69, 77 Moon, 2, 126–28, 129, 138, 174, 203 Moore, Gordon, 42, 55, 56 Moore’s Law, 41–43, 46, 48, 51, 55, 56, 64, 203 Moriarty, James, 85–86 Mount Everest, 140–41 Mueller, John, 165 Munroe, Randall, 84, 153–54 Murphy, Tom, 55 mutation, 87–94 Napier’s constant, 12 National Institutes of Health, 17 natural selection, 104–5, 187 Nature, 122, 154, 156, 162, 166 negative results, 162 Neptune, 154–55, 183 network science, 74–78 neuroscience, 48 New Scientist, 85 Newton, Isaac, 21, 36, 67, 94, 174, 186 New Yorker, 86 New York Times, 20, 75, 174 Nobel laureates, 18 nosebleeds, 180–82 Noyce, Robert, 42 null hypothesis, 152 Obama, Barack, 179 Oliver, John, 159 Onnela, Jukka-Pekka, 69, 77 On the Origin of Species (Darwin), 79, 187 opera, 14–15 orders, 60 Original Theory or New Hypothesis of the Universe, An (Wright), 121–22 Pacioli, Luca, 200 paleography, 87–90 paradigm, 186 paradigm shift, 186, 187 Parmentier, Antoine, 102 particle accelerator, 51 Patent Office, 54 Pauly, Daniel, 172–73 Pepys, Samuel, 52 periodic table, 50, 150–52, 182 Petroski, Henry, 49 phase transitions, 207 in acceptance and assimilation of knowledge, 185, 186 in facts, 121–39, 185 Ising model and, 124, 125–26, 138 in physics, 123–24, 126 Philosophical Transactions of the Royal Society of London, 9, 12 physics, 32 Planck, Max, 186–88 planets, 6, 121–23, 128, 129–31, 132, 183–84 Planet X, 154–56, 160 Pluto, 122–23, 128, 138, 148–49, 155, 183–84 polio, 52 Pony Express, 70 Poovey, Mary, 200 Popeye the Sailor, 83, 213 population: innovation and, 135–37, 202 makeup of, 61 size of, 2, 6, 57–61, 122, 135–37, 204 Portugal, 207 posterior probability, 159 potatoes, 102 preferential attachment, 103 prefixes, 47–48, 142, 147 Price, Derek J. de Solla, 9, 12–13, 15, 17, 32, 47, 50, 103, 166–67 prices, 196–97 printing press, 70–74, 78, 115 prior probability, 159 Pritchett, Lant, 186 Prize4Life Foundation, 97–98 productivity, 55–56 programmed cell death, 111, 194 proteomics, 48 Proteus phenomenon, 161 publication bias, 156 p-values, 152–54, 156, 158 P versus NP, 133–35 “Quantitative Measures of the Development of Science” (Price), 12 Quebec, 193–94 Queloz, Didier, 122 radioactivity, 2–3, 29, 33 Raynaud’s syndrome, 99, 110 reading, 197–98 Real Time Statistics Project, 195 reinventions, 104–5 Rendezvous with Rama (Clarke), 19 Rényi, Alfréd, 104 replication, 161–62 Riggs, Elmer, 81 Robinson, Karen, 107–8 robots, 46 Royal Society, 94–95 Roychowdhury, Vwani, 91, 103–4 Russell, C.

., 81–82 McWhorter, John, 191 measurement, 142–70 decline effect and, 155–56, 157 kilogram in, 147–48 meter in, 143–47 of Mount Everest, 140–41 precision and accuracy in, 149–50 prefixes in, 47–48, 142, 147 publication bias and, 156 of trees, 142 Mechanical Turk, 180–82 medical knowledge, 23, 32, 51–52, 53, 122, 197, 198, 208 about cirrhosis and hepatitis, 28–30 MEDLINE, 99–100 memorization, 198 Mendel, Gregor, 106 Mendeley, 117, 118 Merton, Robert, 61, 103, 104 mesofacts, 6–7, 195, 203 meta-analysis, 107–8 cumulative, 109–10 meter, 143–47 Milgram, Stanley, 24, 167 mobile phone calls, 69, 77 Moon, 2, 126–28, 129, 138, 174, 203 Moore, Gordon, 42, 55, 56 Moore’s Law, 41–43, 46, 48, 51, 55, 56, 64, 203 Moriarty, James, 85–86 Mount Everest, 140–41 Mueller, John, 165 Munroe, Randall, 84, 153–54 Murphy, Tom, 55 mutation, 87–94 Napier’s constant, 12 National Institutes of Health, 17 natural selection, 104–5, 187 Nature, 122, 154, 156, 162, 166 negative results, 162 Neptune, 154–55, 183 network science, 74–78 neuroscience, 48 New Scientist, 85 Newton, Isaac, 21, 36, 67, 94, 174, 186 New Yorker, 86 New York Times, 20, 75, 174 Nobel laureates, 18 nosebleeds, 180–82 Noyce, Robert, 42 null hypothesis, 152 Obama, Barack, 179 Oliver, John, 159 Onnela, Jukka-Pekka, 69, 77 On the Origin of Species (Darwin), 79, 187 opera, 14–15 orders, 60 Original Theory or New Hypothesis of the Universe, An (Wright), 121–22 Pacioli, Luca, 200 paleography, 87–90 paradigm, 186 paradigm shift, 186, 187 Parmentier, Antoine, 102 particle accelerator, 51 Patent Office, 54 Pauly, Daniel, 172–73 Pepys, Samuel, 52 periodic table, 50, 150–52, 182 Petroski, Henry, 49 phase transitions, 207 in acceptance and assimilation of knowledge, 185, 186 in facts, 121–39, 185 Ising model and, 124, 125–26, 138 in physics, 123–24, 126 Philosophical Transactions of the Royal Society of London, 9, 12 physics, 32 Planck, Max, 186–88 planets, 6, 121–23, 128, 129–31, 132, 183–84 Planet X, 154–56, 160 Pluto, 122–23, 128, 138, 148–49, 155, 183–84 polio, 52 Pony Express, 70 Poovey, Mary, 200 Popeye the Sailor, 83, 213 population: innovation and, 135–37, 202 makeup of, 61 size of, 2, 6, 57–61, 122, 135–37, 204 Portugal, 207 posterior probability, 159 potatoes, 102 preferential attachment, 103 prefixes, 47–48, 142, 147 Price, Derek J. de Solla, 9, 12–13, 15, 17, 32, 47, 50, 103, 166–67 prices, 196–97 printing press, 70–74, 78, 115 prior probability, 159 Pritchett, Lant, 186 Prize4Life Foundation, 97–98 productivity, 55–56 programmed cell death, 111, 194 proteomics, 48 Proteus phenomenon, 161 publication bias, 156 p-values, 152–54, 156, 158 P versus NP, 133–35 “Quantitative Measures of the Development of Science” (Price), 12 Quebec, 193–94 Queloz, Didier, 122 radioactivity, 2–3, 29, 33 Raynaud’s syndrome, 99, 110 reading, 197–98 Real Time Statistics Project, 195 reinventions, 104–5 Rendezvous with Rama (Clarke), 19 Rényi, Alfréd, 104 replication, 161–62 Riggs, Elmer, 81 Robinson, Karen, 107–8 robots, 46 Royal Society, 94–95 Roychowdhury, Vwani, 91, 103–4 Russell, C.

pages: 321 words: 97,661

How to Read a Paper: The Basics of Evidence-Based Medicine by Trisha Greenhalgh


call centre, complexity theory, conceptual framework, correlation coefficient, correlation does not imply causation, deskilling, knowledge worker, meta analysis, meta-analysis, microbiome, New Journalism, p-value, personalized medicine, placebo effect, publication bias, randomized controlled trial, selection bias, the scientific method

Remember, too, that the results of an RCT may have limited applicability as a result of exclusion criteria (rules about who may not be entered into the study), inclusion bias (selection of trial participants from a group that is unrepresentative of everyone with the condition (see section ‘Whom is the study about?’)), refusal (or inability) of certain patient groups to give consent to be included in the trial, analysis of only pre-defined ‘objective’ endpoints which may exclude important qualitative aspects of the intervention (see Chapter 12) and publication bias (i.e. the selective publication of positive results, often but not always because the organisation that funded the research stands to gain or lose depending on the findings [9] [10]). Furthermore, RCTs can be well or badly managed [2], and, once published, their results are open to distortion by an over-enthusiastic scientific community or by a public eager for a new wonder drug [13]. While all these problems might also occur with other trial designs, they may be particularly pertinent when an RCT is being sold to you as, methodologically speaking, whiter than white.

The authors report a series of artificial dice-rolling experiments in which red, white and green dice, respectively, represented different therapies for acute stroke. Overall, the ‘trials’ showed no significant benefit from the three therapies. However, the simulation of a number of perfectly plausible events in the process of meta-analysis—such as the exclusion of several of the ‘negative’ trials through publication bias (see section ‘Randomised controlled trials’), a subgroup analysis that excluded data on red dice therapy (because, on looking back at the results, red dice appeared to be harmful), and other, essentially arbitrary, exclusions on the grounds of ‘methodological quality’—led to an apparently highly significant benefit of ‘dice therapy’ in acute stroke. You cannot, of course, cure anyone of a stroke by rolling a dice, but if these simulated results pertained to a genuine medical controversy (such as which postmenopausal women would be best advised to take hormone replacement therapy or whether breech babies should routinely be delivered by Caesarean section), how would you spot these subtle biases?

Eysenck's reservations about meta-analysis are borne out in the infamously discredited meta-analysis that demonstrated (wrongly) that there was significant benefit to be had from giving intravenous magnesium to heart attack victims. A subsequent megatrial involving 58 000 patients (ISIS-4) failed to find any benefit whatsoever, and the meta-analysts' misleading conclusions were subsequently explained in terms of publication bias, methodological weaknesses in the smaller trials and clinical heterogeneity [22] [23]. (Incidentally, for more debate on the pros and cons of meta-analysis versus megatrials, see this recent paper [24].) Eysenck's mathematical naiveté is embarrassing (‘if a medical treatment has an effect so recondite and obscure as to require a meta-analysis to establish it, I would not be happy to have it used on me’), which is perhaps why the editors of the second edition of the ‘Systematic reviews’ book dropped his chapter from their collection.

pages: 128 words: 35,958

Getting Back to Full Employment: A Better Bargain for Working People by Dean Baker, Jared Bernstein


2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, Affordable Care Act / Obamacare, American Society of Civil Engineers: Report Card, Asian financial crisis, collective bargaining, declining real wages, full employment, George Akerlof, income inequality, inflation targeting, mass immigration, minimum wage unemployment, new economy, price stability, publication bias, quantitative easing, Report Card for America’s Infrastructure, rising living standards, selection bias, War on Poverty

While there are plenty of examples of countries that have maintained healthy growth rates even with double-digit inflation rates, it is fair to say that such rates raise a qualitatively different set of questions than the inflation rates that may arise from having an unemployment rate that is 0.5 to 1.0 percentage points below the level that is consistent with stable inflation for a period of time. [23] In fairness to advocates of inflation targeting, there is a wide range of views as to how strictly we should hold to the target as the primary or only goal of monetary policy. [24] There is also the possibility of publication bias. Given the strong belief by many economists that inflation reduces growth, there may be a reluctance to publish articles that find either insignificant results or even a positive relationship. This sort of publication bias was noted in the case of the minimum wage, where the distribution of published results has an otherwise inexplicable break at zero. If we assume that study results are normally distributed, there should be some number of studies that find a significant positive relationship between higher minimum wages and employment even if the true coefficient for an employment variable is zero (Doucouliagos and Stanley 2009)

Everydata: The Misinformation Hidden in the Little Data You Consume Every Day by John H. Johnson


Affordable Care Act / Obamacare, Black Swan, business intelligence, Carmen Reinhart, cognitive bias, correlation does not imply causation, Daniel Kahneman / Amos Tversky, Donald Trump,, Kenneth Rogoff, labor-force participation, lake wobegon effect, Long Term Capital Management, Mercator projection, Mercator projection distort size, especially Greenland and Africa, meta analysis, meta-analysis, Nate Silver, obamacare, p-value, PageRank, pattern recognition, publication bias, QR code, randomized controlled trial, risk-adjusted returns, Ronald Reagan, selection bias, statistical model, The Signal and the Noise by Nate Silver, Thomas Bayes, Tim Cook: Apple, wikimedia commons, Yogi Berra

“P‑hacking” (named after p‑values) is a term used when researchers “collect or select data or statistical analyses until nonsignificant results become significant,” according to a PLoS Biology article.”36 This is similar to cherry picking, as p‑hacking researchers simply throw things at the wall until something sticks, metaphorically speaking (although there probably are some scientists who actually throw things at the wall until something sticks . . . ). 221158 i-xiv 1-210 r4ga.indd  79 2/8/16  5:58:50 PM 80 E V E R Y D ATA n n A fascinating New Yorker article (is there any other kind?) examines publication bias as a possible cause of the “decline effect,” in which the size of a statistically significant effect declines over time. Why? One statistician found that “­ninety-​­seven per cent of all published psychological studies with statistically significant data found the effect they were looking for,” making it perhaps less likely that future studies would be able to replicate these results.37 The Journal of Epidemiology and Community Health published a paper finding no evidence that reduced street lighting at night increased traffic collisions or crime in England and Wales.

See also misrepresentation and misinterpretation brain’s hardwiring for, 60–61 challenges in, 54–55 Ioannidis, John, 75 iPhones, 46–48, 58 “Ipse dixit” bias, 94 J Japan earthquake of 2011, 123–125 Jordan, Michael, 53 Journal of Epidemiology and Community Health, 80 Journal of Finance, 139–140 Journal of Safety Research, 20 Journal of the American Medical Informatics Association, 148 Journal of the National Cancer Institute, 69–70 K Katz, David, 22 Keillor, Garrison, 43 221158 i-xiv 1-210 r4ga.indd  197 197 L Lake Wobegon effect, 42–43 Landon, Alfred, 132 Law360, 146–148 Lawyer Satisfaction Survey, 146–148 Literary Digest, 132 longevity, 4, 87–92 Los Angeles Times, 17–18 Lotto Stats, 133 Lund, Bob, 10 M magnitude, 77–78, 81 in birth month and health study, 149 map projections, 83–85 margins of error, 38, 68–69 Marie Claire, 34–35 math mistakes, 101–102, 103 mayors/deputy mayors salaries, 35–36 McCarthy, Jenny, 61 McGwire, Mark, 39 meaning, difficulty of extracting from too much data, 4. See also misrepresentation and misinterpretation means, 32–34 definition of, 32 mean trimming, 40 media cherry-picking by, 116 data interpretation by, 75, 81 publication bias and, 80 medians, 32–34 definition of, 32 medical coding errors, 97 Medical News Today, 75 memory of printed vs. online material, 2 Mercator, Gerardus, 83–85 2/8/16  5:58:50 PM 198 Index misrepresentation and misinterpretation, 83–103. See also cherry-picking in charts, 87–92 correlation/causation based on, 58–60 data sources and, 99 errors and, 97–99 of food expiration dates, 99–100 in gas tank gauges, 96–97 guessing and, 86 helpful, 96–97 how to be a smart consumer and, 102–103 math mistakes and, 101–102 in the media, 75, 81 “only” and, 95–96 from treating all data equally, 95 trust in expertise and, 93–94 with visuals, 92–94 models, forecasts based on, 125–127 modes, 32–34 definition of, 32 Moore, Michael, 116 Morton Thiokol, 10, 55 multiple comparison problem, 75–76 N National Bureau of Economic Research, 59, 69 National Cancer Institute, 69–70 National Electronic Injury Surveillance System (NEISS), 18 National Foundation for Celiac Awareness, 21 National Weight Control Registry (NWCR), 17 Natural Resources Defense Council, 100 Nest, 100–101 Newman, Mark, 28–29 New York State Office of the Attorney General, 97 221158 i-xiv 1-210 r4ga.indd  198 New York Times, 66–67 New York Times Magazine, 101 Nielsen, Arthur, Sr., 25 Nike, 53 NPD Group, 21 NWEA Measures of Academic Progress (MAP), 22–23 O Obama, Barack, 23, 27–30 observations, definition of, 13.

., 58, 76, 135 2/8/16  5:58:50 PM Index presidential campaigns/elections averages/aggregates and, 27–30, 44 cherry-picking in, 115–116 forecasting, 132, 137 polls and, 37–38, 68–69, 73 sampling and, 20 terms of office and, 41 Princeton Review of schools, 19 printed material vs. online differences in consumption/ interpretation of, 7 willingness to question, 93–94 printed vs. online material memory of, 2 probability, 70–71, 81 coincidence and, 138–139 forecasting and, 131 proxies, 49–50 psychology research, 15–16 publication bias, 80 p‑values, 71, 72, 79 Q questions/questioning, 7–8 cherry-picking and, 122 correlation vs. causation, 60 of print vs. online information, 93–94 quote mining, 116 R Radio Television Digital News Association, 36 random chance, multiple comparison problem and, 75–76, 80–81 random samples, 65–68 Rate My Professor, 51–52 Reagan, Ronald, 9 recall of printed vs. online material, 2 Reinhart, Carmen, 97–98 relationships, 5–6.

pages: 436 words: 123,488

Overdosed America: The Broken Promise of American Medicine by John Abramson


germ theory of disease, Louis Pasteur, medical malpractice, medical residency, meta analysis, meta-analysis, p-value, placebo effect, profit maximization, profit motive, publication bias, RAND corporation, randomized controlled trial, selective serotonin reuptake inhibitor (SSRI), stem cell, Thomas Kuhn: the structure of scientific revolutions

The article also reported that 92 percent of these ads were in some violation of FDA rules. Writing in The Lancet in 2003, Dr. Fletcher said that as punishment for publishing this article, the pharmaceutical industry “withdrew many adverts” and showed that it was “willing to flex its considerable muscles when it felt its interests were threatened.” This is a price that medical journal editors would prefer not to pay. NOT TELLING THE WHOLE TRUTH: PUBLICATION BIAS Even if a doctor could keep up with all the studies that were published, he or she would still have a limited and skewed view of the real evidence. Notwithstanding all the potential ways that research can be tipped in favor of a sponsor’s product, clinical trials still tend to reveal the truth about whether a new therapy is effective—or not. The problem is that research that shows that a product is not effective or safe can be hidden away, that is, the “knowledge” can be filtered to let through findings that favor the sponsor’s product, making it difficult for even the most fastidious doctors to discover the truth.

In another study, researchers in the United States obtained data under the Freedom of Information Act from all of the studies (both published and unpublished) that the FDA had reviewed in the process of approving seven new antidepressants (Prozac, Zoloft, Paxil, Effexor, Serzone, Remeron, and Wellbutrin SR) between 1987 and 1997—a total of 5200 pages of documents. The results of all of the “pivotal” studies (those deemed to be of high enough quality to be used in the FDA’s determinations) for these seven antidepressants were then put together to assess the overall effect of the new drugs. By looking at all the studies, the researchers avoided the distortion of “publication bias” and were able to determine whether or not the scientific evidence really showed that the new antidepressants are more effective and safer than the older ones. When all the evidence is considered, it turns out that the new antidepressant drugs are no more effective than the older tricyclic antidepressants (the classic being amitriptyline, brand name Elavil). More important, the new antidepressants were found to be not even 10 percent more effective than the placebos: Symptoms of depression improved by 30.9 percent in the people who took the placebos; by 40.7 percent in the people who took the newer antidepressants; and by 41.7 percent in the people who took the older antidepressants.

See also medical research absolute vs. relative risk and, 14–16, 165, 166, 229 advertising and research companies and, 109–10 Celebrex and Vioxx research (see Celebrex and Vioxx) cholesterol research (see cholesterol guidelines of 2001) commercial funding, 94–97 (see also drug companies; funding) commercial goals vs. health goals, 21–22, 50–51, 53, 241–44 conflicts of interest and (see conflicts of interest) damage control and, 107–9 data manipulation, 34–36 data omission, 29–31 data transparency and, 27–28, 94, 105–6, 251–52 dosage manipulation, 101–2 failure to compare existing therapies, 17, 102–3 FDA drug approval and Rezulin, 86–88 ghostwriters and, 106–7 hormone replacement therapy (see hormone replacement therapy) implantable defibrillators, 98–101 independent review for, 249–53 medical journals and, 25–27, 37–38, 93–94, 96–97 (see also medical journals) osteoporosis research, 211–20 Paxil research, 243 premature termination of research, 104–5 publication bias as, 113–17 research design changes as, 31 septic shock research, 161–63 stroke research, 13–22 unbiased information vs., 167 unrepresentative patients, 16–17, 33, 103–4, 206–8, 251 commercial speech, 37–38, 157–59 conflicts of interest academic experts, xxii, 18, 243 cholesterol guidelines, 135, 147–48 clinical guideline experts, xxi, 127–28, 133–35, 146–48, 227, 249–50 continuing medical education, 121–23 damage control, 109 FDA, 85–87, 89–90 ghostwriters, 106–7 hormone replacement therapy, 60–61 medical journal, 26 medical news stories, 166–67 NIH researchers, 86–90 independent review and, 258–59 surgeons, 177–78 confounding factors, 66–67 consciousness, 206–8 consulting contracts, 88–90, 109, 249.

pages: 357 words: 110,072

Trick or Treatment: The Undeniable Facts About Alternative Medicine by Edzard Ernst, Simon Singh


Barry Marshall: ulcers, Berlin Wall, correlation does not imply causation, false memory syndrome, Florence Nightingale: pie chart, germ theory of disease, John Snow's cholera map, Louis Pasteur, meta analysis, meta-analysis, placebo effect, profit motive, publication bias, randomized controlled trial, Ronald Reagan, Simon Singh, The Design of Experiments, the scientific method

Therefore, either Eastern researchers or Western researchers must be wrong – as it happens, there are good reasons to believe that the problem lies in the East. The crude reason for blaming Chinese researchers for the discrepancy is that their results are simply too good to be true. This criticism has been confirmed by careful statistical analyses of all the Chinese results, which demonstrate beyond all reasonable doubt that Chinese researchers are guilty of so-called publication bias. Before explaining the meaning of publication bias, it is important to stress that this is not necessarily a form of deliberate fraud, because it is easy to conceive of situations when it can occur due to an unconscious pressure to get a particular result. Imagine a Chinese researcher who conducts an acupuncture trial and achieves a positive result. Acupuncture is a major source of prestige for China, so the researcher quickly and proudly publishes his positive result in a journal.

The key point is that this second piece of research might never be published for a whole range of possible reasons: maybe the researcher does not see it as a priority, or he thinks that nobody will be interested in reading about a negative result, or he persuades himself that this second trial must have been badly conducted, or he feels that this latest result would offend his peers. Whatever the reason, the researcher ends up having published the positive results of the first trial, while leaving the negative results of the second trial buried in a drawer. This is publication bias. When this sort of phenomenon is multiplied across China, then we have dozens of published positive trials, and dozens of unpublished negative trials. Therefore, when the WHO conducted a review of the published literature that relied heavily on Chinese research its conclusion was bound to be skewed – such a review could never take into account the unpublished negative trials. The WHO report was not just biased and misleading; it was also dangerous because it was endorsing acupuncture for a whole range of conditions, some of which were serious, such as coronary heart disease.

pages: 270 words: 79,992

The End of Big: How the Internet Makes David the New Goliath by Nicco Mele


3D printing, 4chan, A Declaration of the Independence of Cyberspace, Airbnb, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, Apple's 1984 Super Bowl advert, barriers to entry, Berlin Wall, big-box store, bitcoin, business climate, call centre, Cass Sunstein, centralized clearinghouse, Chelsea Manning, citizen journalism, cloud computing, collaborative consumption, collaborative editing, commoditize, creative destruction, crony capitalism, cross-subsidies, crowdsourcing, David Brooks, death of newspapers, Donald Trump, Douglas Engelbart, Douglas Engelbart,, Exxon Valdez, Fall of the Berlin Wall, Filter Bubble, Firefox, Galaxy Zoo, global supply chain, Google Chrome, Gordon Gekko, Hacker Ethic, Jaron Lanier, Jeff Bezos, jimmy wales, John Markoff, Julian Assange, Kevin Kelly, Khan Academy, Kickstarter, Lean Startup, Mark Zuckerberg, minimum viable product, Mohammed Bouazizi, Mother of all demos, Narrative Science, new economy, Occupy movement, old-boy network, peer-to-peer, period drama, Peter Thiel, pirate software, publication bias, Robert Metcalfe, Ronald Reagan, Ronald Reagan: Tear down this wall, sharing economy, Silicon Valley, Skype, social web, Steve Jobs, Steve Wozniak, Stewart Brand, Stuxnet, Ted Nelson, Telecommunications Act of 1996, telemarketer, The Wisdom of Crowds, transaction costs, uranium enrichment, Whole Earth Catalog, WikiLeaks, Zipcar

Nielsen and other advocates for “open science” say science can accomplish much more, much faster, in an environment of friction-free collaboration over the Internet.25 Peer review may provide accountability, but it is in many ways deeply flawed and inadequate in the digital age. Peer-reviewed publication takes on average about two years, and many scientific journals costs thousands of dollars a year for subscriptions. Not only that, but if scientific research fails, it usually does not get written up and published. Who wants to publish an article that says, “we tried this and it didn’t work”? “Publication bias” is a well-known challenge in academia. A major review of more than 4,600 peer-reviewed academic papers across a range of disciplines and a range of countries found that over the last twenty years, positive results increased by almost 25%.26 And yet failure is a crucial part of the scientific process. To better figure out what works, you need to know what doesn’t work. Despite the entrenched function of peer review within the academic establishment, online correctives and alternatives have proliferated.

Harry Lewis, Excellence Without a Soul: How a Great University Forgot Education (New York: PublicAffairs, 2006), 8. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. David Weinberger, Too Big to Know: Rethinking Knowledge Now That the Facts Aren’t the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room (New York: Basic Books, 2011), 139. 29. Ibid., 140. 30. 31. 32. 34. 35.

pages: 586 words: 159,901

Wall Street: How It Works And for Whom by Doug Henwood


accounting loophole / creative accounting, activist fund / activist shareholder / activist investor, affirmative action, Andrei Shleifer, asset allocation, asset-backed security, bank run, banking crisis, barriers to entry, borderless world, Bretton Woods, British Empire, capital asset pricing model, capital controls, central bank independence, computerized trading, corporate governance, corporate raider, correlation coefficient, correlation does not imply causation, credit crunch, currency manipulation / currency intervention, David Ricardo: comparative advantage, debt deflation, declining real wages, deindustrialization, dematerialisation, diversification, diversified portfolio, Donald Trump, equity premium, Eugene Fama: efficient market hypothesis, experimental subject, facts on the ground, financial deregulation, financial innovation, Financial Instability Hypothesis, floating exchange rates, full employment, George Akerlof, George Gilder, hiring and firing, Hyman Minsky, implied volatility, index arbitrage, index fund, information asymmetry, interest rate swap, Internet Archive, invisible hand, Irwin Jacobs, Isaac Newton, joint-stock company, Joseph Schumpeter, kremlinology, labor-force participation, late capitalism, law of one price, liberal capitalism, liquidationism / Banker’s doctrine / the Treasury view, London Interbank Offered Rate, Louis Bachelier, market bubble, Mexican peso crisis / tequila crisis, microcredit, minimum wage unemployment, money market fund, moral hazard, mortgage debt, mortgage tax deduction, Myron Scholes, oil shock, Paul Samuelson, payday loans, pension reform, Plutocrats, plutocrats, price mechanism, price stability, prisoner's dilemma, profit maximization, publication bias, Ralph Nader, random walk, reserve currency, Richard Thaler, risk tolerance, Robert Gordon, Robert Shiller, Robert Shiller, selection bias, shareholder value, short selling, Slavoj Žižek, South Sea Bubble, The inhabitant of London could order by telephone, sipping his morning tea in bed, the various products of the whole earth, The Market for Lemons, The Nature of the Firm, The Predators' Ball, The Wealth of Nations by Adam Smith, transaction costs, transcontinental railway, women in the workforce, yield curve, zero-coupon bond

But in surveys of employers taken just before and after changes in the minimum wage, David Card and Alan Krueger (1995) showed that this just isn't true. They paused for a few pages in the middle of their book. Myth and Measurement, to review some reasons why the academic literature has almost unanimously found the minimum wage guilty as charged. They surmised that earlier studies showing that higher wages reduced employment were the result of "publication bias" among journal editors. They also surmised, very diplomatically, that economists have been aware of this bias, and played those notorious scholarly games, "specification searching and data mining" — bending the numbers to obtain the desired result. They also noted that some of the early studies were based on seriously flawed data, but since the results were desirable from both the political and professional points of view, they went undiscovered for several years.

See money managers portfolio vs. direct investment, 109 Post Keynesian Thought (PKT) computer network, 243 post-Keynesianism, 217-224 defined, 241-242 see also money, endogenous postmodernism. 237, 245 present value, 119-120 priest, banker's advice more useful than, 225 primitive accumulation, 252 prisoners' dilemma, 171, 183 Pritzker family, 271 private placements, 75 privatization, 110 of economic statistics, I36 returning capital flight and, 295 Social Security, 303-307 production, 241 socialization of, 240 productivity, 299 failure to boom in 1980s, 183 profit(s) maximization Galbraith on, 259 Herman on, 260 and modern corporation, 254 transformation into interest, 73-74, 238 Progressive Era, 94 property relations and social investing, 314- 315 prostitutes. Wail Streeters as customers, 79 protectionism, 295, 300 Proudhonism, 301 The Prudential, 262 investigations of, 304 psychoanalysis, 315 psychology and stock prices, 176-178; see also Keynes, John Maynard; money, psychology of public goods, 143 public relations, ll6 publication bias, 141 Pujo Committee, 260 Pulitzer Prize, 298 puritans of finance, 196 puts, 30; see also derivatives q ratio and capital expenditures, 145-148 and LBOs, 283 and M&A, 148, 284, 299 as stock market predictors, 148 Quan, Tracy, 79 race financial workers, 78 and wealth distribution, 69-70 racism, 98 among goldbugs, 48 Keynes's, 212 railroads and modern corporation, 188 Rainforest Crunch, 313 Rand, Ayn, 47, 89 random walk, 164 Rathenau, Walther, 256 rational expectations, l6l; see fl&o efficient market theory rationality, assumption of, 175 Ravenscraft, David, 279, 283-284 Reagan, Ronald, 87 real estate, 80 real sector predicting the financial, 125-126; see also business cycles Reconstruction Finance Corp., 286 reform, financial, difficulty of, 302 Regan, Edward, 27 regulation, government, overview, 90-99 Reich, Robert, 131 Relational Investors, 289 relationship investing, 293 religion banking and, 225 and belief in markets, 150 monetarism as, 242 and money, 225 restrictions on usury, 42 rentiers apologists, 293 appropriation of worker savings, 239 capture of Clinton administration, 134 consciousness, 237, 238; see also money, psychology of corporate cash flow share, 73-74 dominance of political discourse, 294 early 1990s riot, 288-291 euthanasia of, 210 evolutionary role, 8 formation through financial markets, 238 growing assertiveness, 207 proliferate over time, 215, 236 who needs them?

pages: 173 words: 14,313

Peers, Pirates, and Persuasion: Rhetoric in the Peer-To-Peer Debates by John Logie


1960s counterculture, Berlin Wall, book scanning, cuban missile crisis, Fall of the Berlin Wall, Hacker Ethic, Isaac Newton, Marshall McLuhan, moral panic, mutually assured destruction, peer-to-peer, Plutocrats, plutocrats, pre–internet, publication bias, Richard Stallman, search inside the book, SETI@home, Silicon Valley, slashdot, Steve Jobs, Steven Levy, Stewart Brand, Whole Earth Catalog

., copyrights and patents) are offered by the people, via Congress, and for the people, as an incentive for further production from authors and inventors. This represents a subtle but significant break from a broader European tradition in which the so-called “natural rights” of the author or inventor function as the bases for intellectual property protections. The 1991 Supreme Court’s ringing endorsement of copyright’s inherent public bias in the Feist case (once again: “The primary objective of copyright is not to reward the labor of authors, but ‘[t]o promote the Progress of Science and useful Arts.’”) almost certainly emboldened Robertson as he set about developing the service. Robertson even agreed with the RIAA that Napster was enabling piracy. As the Napster case was still wending its way through the courts, Robertson was sharply critical of his fellow RIAA defendant.

pages: 266 words: 67,272

Fun Inc. by Tom Chatfield


Alexey Pajitnov wrote Tetris, Any sufficiently advanced technology is indistinguishable from magic, cloud computing, cognitive dissonance, computer age, credit crunch, game design, invention of writing, moral panic, publication bias, Silicon Valley, Skype, stem cell, upwardly mobile

Its author, Dr Christopher John Ferguson, an assistant professor of psychology at Texas A&M International University, set out to compare every article published in a peer-reviewed journal between 1995 and April 2007 that in some way investigated the effect of playing violent video games on some measure of aggressive behaviour. A total of seventeen published studies matched these criteria – and Ferguson’s conclusions were unexpectedly unequivocal. ‘Once corrected for publication bias,’ he reported, ‘studies of video game violence provided no support for the hypothesis that violent video game-playing is associated with higher aggression.’ Moreover, he added, the question ‘do violent games cause violence?’ is itself flawed in that ‘it assumes that such games have only negative effects and ignores the possibility of positive effects’ such as the possibility that violent games allow ‘catharsis’ of a kind in their players.

Grain Brain: The Surprising Truth About Wheat, Carbs, and Sugar--Your Brain's Silent Killers by David Perlmutter, Kristin Loberg


epigenetics, Gary Taubes, meta analysis, meta-analysis, microbiome, mouse model, phenotype, publication bias, Ralph Waldo Emerson, selective serotonin reuptake inhibitor (SSRI), stem cell

In 2010, the American Journal of Clinical Nutrition published an astonishing study that revealed the truth behind urban legends about fat, especially the saturated kind, and heart disease.14 The study was a retrospective evaluation of twenty-one previous medical reports involving more than three hundred forty thousand subjects followed from periods of five to twenty-three years. It concluded that “intake of saturated fat was not associated with an increased risk of coronary heart disease, stroke, or cardiovascular disease.” In comparing the lowest to the highest consumption of saturated fat, the actual risk for coronary heart disease was 19 percent lower in the group consuming the highest amount of saturated fat. The authors also stated: “Our results suggested a publication bias, such that studies with significant associations tended to be received more favorably for publication.” What the authors are implying is that when other studies presented conclusions that were more familiar to the mainstream (i.e., fat causes heart disease), not to mention more attractive to Big Pharma, they were more likely to get published. The truth is we thrive on saturated fats. In the words of Michael Gurr, PhD, author of Lipid Biochemistry: An Introduction, “Whatever causes coronary heart disease, it is not primarily a high intake of saturated fatty acids.”15 In a subsequent report from the American Journal of Clinical Nutrition, a panel of leading researchers in the field of nutrition from around the globe clearly stated: “At present there is no clear relation of saturated fatty acid intake to these outcomes [of obesity, cardiovascular disease, incidence of cancer and osteoporosis].”

pages: 274 words: 93,758

Phishing for Phools: The Economics of Manipulation and Deception by George A. Akerlof, Robert J. Shiller, Stanley B Resor Professor Of Economics Robert J Shiller


Andrei Shleifer, asset-backed security, Bernie Madoff, Capital in the Twenty-First Century by Thomas Piketty, collapse of Lehman Brothers, corporate raider, Credit Default Swap, Daniel Kahneman / Amos Tversky, dark matter, David Brooks,, endowment effect, equity premium, financial intermediation, financial thriller, fixed income, full employment, George Akerlof, greed is good, income per capita, invisible hand, John Maynard Keynes: Economic Possibilities for our Grandchildren, Kenneth Arrow, Kenneth Rogoff, late fees, loss aversion, Menlo Park, mental accounting, Milgram experiment, money market fund, moral hazard, new economy, Pareto efficiency, Paul Samuelson, payday loans, Ponzi scheme, profit motive, publication bias, Ralph Nader, randomized controlled trial, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Reagan, Silicon Valley, the new new thing, The Predators' Ball, the scientific method, The Wealth of Nations by Adam Smith, theory of mind, Thorstein Veblen, too big to fail, transaction costs, Unsafe at Any Speed, Upton Sinclair, Vanguard fund, Vilfredo Pareto, wage slave

Gross, “Scope and Impact of Financial Conflicts of Interest in Biomedical Research: A Systematic Review,” Journal of the American Medical Association 289, no. 4 (January 22, 2003): 454–65; Joel Lexchin, Lisa A. Bero, Benjamin Djulbegovic, and Otavio Clark, “Pharmaceutical Industry Sponsorship and Research Outcome and Quality: Systematic Review,” British Medical Journal 326, no. 7400 (May 31, 2003): 1167. Bekelman, Li, and Gross also refer to two studies of “multiple reporting of studies with positive outcomes, further compounding publication bias.” 17. Bob Grant, “Elsevier Published 6 Fake Journals,” The Scientist, May 7, 2009, accessed November 24, 2014, display/55679/. See also Ben Goldacre, Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients (New York: Faber and Faber / Farrar, Straus and Giroux, 2012), pp. 309–10. 18. Claire Bombardier et al., “Comparison of Upper Gastrointestinal Toxicity of Rofecoxib and Naproxen in Patients with Rheumatoid Arthritis,” New England Journal of Medicine 343, no. 21 (November 23, 2000): 1520–28. 19.

pages: 410 words: 114,005

Black Box Thinking: Why Most People Never Learn From Their Mistakes--But Some Do by Matthew Syed


Airbus A320, Alfred Russel Wallace, Arthur Eddington, Atul Gawande, Black Swan, British Empire, call centre, Captain Sullenberger Hudson, Checklist Manifesto, cognitive bias, cognitive dissonance, conceptual framework, corporate governance, creative destruction, credit crunch, crew resource management, deliberate practice, double helix, epigenetics, fear of failure, fundamental attribution error, Henri Poincaré, hindsight bias, Isaac Newton, iterative process, James Dyson, James Hargreaves, James Watt: steam engine, Joseph Schumpeter, Lean Startup, mandatory minimum, meta analysis, meta-analysis, minimum viable product, publication bias, quantitative easing, randomized controlled trial, selection bias, Silicon Valley, six sigma, spinning jenny, Steve Jobs, the scientific method, Thomas Kuhn: the structure of scientific revolutions, too big to fail, Toyota Production System, US Airways Flight 1549, Wall-E, Yom Kippur War

As Anthony Hidden QC, the man who investigated the Clapham Junction Rail Disaster, which killed thirty-five people in 1988, put it: “There is almost no human action or decision that cannot be made to look flawed and less sensible in the misleading light of hindsight.” *This has a rather obvious analog with what is sometimes called “defensive medicine,” in which clinicians use a host of unnecessary tests that protect their backs, but massively increase health-care costs. *Science is not without flaws, and an eye should always be kept on social and institutional obstacles to progress. Current concerns include publication bias (whereonly successful experiments are published in journals), the weakness of the peer review system, and the fact that many experiments do not appear to be replicable. For a good review of the issues, see: *As the creativity researcher Charlan Nemeth has put it: “The presence of dissenting minority views appears to stimulate more originality.”

The White Man's Burden: Why the West's Efforts to Aid the Rest Have Done So Much Ill and So Little Good by William Easterly


airport security, anti-communist, Asian financial crisis, bank run, banking crisis, Bob Geldof, Bretton Woods, British Empire, call centre, clean water, colonial exploitation, colonial rule, Edward Glaeser, end world poverty, European colonialism, failed state, farmers can use mobile phones to check market prices, George Akerlof, Gunnar Myrdal, Hernando de Soto, income inequality, income per capita, Indoor air pollution, invisible hand, Kenneth Rogoff, laissez-faire capitalism, land reform, land tenure, Live Aid, microcredit, moral hazard, Naomi Klein, publication bias, purchasing power parity, randomized controlled trial, Ronald Reagan, Scramble for Africa, structural adjustment programs, The Fortune at the Bottom of the Pyramid, the scientific method, The Wealth of Nations by Adam Smith, transaction costs, War on Poverty, Xiaogang Anhui farmers

For the announcement of the Millennium Challenge Corporation on November 26, 2002, see For the quoted passage on the motivation behind this new aid, see http://www.>. 11. 12.Esther Duflo and Michael Kremer, “Use of Randomization in the Evaluation of Development Effectiveness,” mimeograph, Harvard and MIT (2003), discuss publication bias. A classic paper on this problem is J. Bradford DeLong and Kevin Lang, “Are All Economic Hypotheses False?” Journal of Political Economy 100, no. 6 (December 1992): 1257–72. 13.UN Millennium Project Report, “Investing in Development: A Practical Plan to Achieve the Millennium Development Goals,” overview, box 8, p. 41. 14.Commission for Africa, “Our Common Interest: Report of the Commission for Africa,” p. 348; 15.Raghuram G.

pages: 1,199 words: 332,563

Golden Holocaust: Origins of the Cigarette Catastrophe and the Case for Abolition by Robert N. Proctor


bioinformatics, carbon footprint, clean water, corporate social responsibility, Deng Xiaoping, desegregation, facts on the ground, friendly fire, germ theory of disease, index card, Indoor air pollution, information retrieval, invention of gunpowder, John Snow's cholera map, language of flowers, life extension, New Journalism, optical character recognition, pink-collar, Ponzi scheme, Potemkin village, publication bias, Ralph Nader, Ronald Reagan, selection bias, speech recognition, stem cell, telemarketer, Thomas Kuhn: the structure of scientific revolutions, Triangle Shirtwaist Factory, Upton Sinclair, Yogi Berra

Switzer denounced the EPA’s report as highly flawed and “problematic,” peppering his critique with pejoratives like “astonishing,” “equivocal,” “deceptive and pointless,” and “serious difficulties.” The Stanford statistician accused the EPA of imprecision, inconsistency, faulty interpretations, improper extrapolations, use of “crude and disputable” estimates of exposure, bias from confounding and misclassification, improper treatment of publication bias, reliance on inconsistent or improperly recorded data, and several other flaws.39 Switzer was well paid for his services, receiving a total of $647,046 from CIAR and other grants in one two-year period. He was also paid handsomely for private consultations with cartel law firms. In one three-month period in the fall of 1991 he received $26,900 from Covington & Burling for consulting on “health effects of exposure to ETS in the workplace” and an analysis of “epidemiology of spousal smoke exposure and lung cancer.”

pages: 1,261 words: 294,715

Behave: The Biology of Humans at Our Best and Worst by Robert M. Sapolsky

autonomous vehicles, Bernie Madoff, biofilm, blood diamonds, British Empire, Broken windows theory, Brownian motion, car-free, clean water, cognitive dissonance, corporate personhood, corporate social responsibility, Daniel Kahneman / Amos Tversky, delayed gratification, desegregation, double helix, Drosophila, Edward Snowden,, epigenetics, Flynn Effect, framing effect, fudge factor, George Santayana, hiring and firing, illegal immigration, impulse control, income inequality, John von Neumann, Loma Prieta earthquake, long peace, loss aversion, Mahatma Gandhi, meta analysis, meta-analysis, Mohammed Bouazizi, mouse model, mutually assured destruction, Network effects, out of africa, Peter Singer: altruism, phenotype, placebo effect, publication bias, RAND corporation, risk tolerance, Rosa Parks, selective serotonin reuptake inhibitor (SSRI), self-driving car, Silicon Valley, stem cell, Steven Pinker, strikebreaker, theory of mind, transatlantic slave trade, traveling salesman, trickle-down economics, ultimatum game, Walter Mischel, wikimedia commons, zero-sum game

., “Television Viewing and Aggressive Behavior During Adolescence and Adulthood,” Sci 295 (2002): 2468; J. Savage and C. Yancey, “The Effects of Media Violence Exposure on Criminal Aggression: A Meta-analysis,” Criminal Justice and Behav 35 (2008): 772; C. Anderson et al., “Violent Video Game Effects on Aggression, Empathy, and Prosocial Behavior in Eastern and Western Countries: A Meta-analytic Review,” Psych Bull 136, 151; C. J. Ferguson, “Evidence for Publication Bias in Video Game Violence Effects Literature: A Meta-analytic Review,” Aggression and Violent Behavior 12 (2007): 470; C. Ferguson, “The Good, the Bad and the Ugly: A Meta-analytic Review of Positive and Negative Effects of Violent Video Games,” Psychiatric Quarterly 78 (2007): 309. 42. W. Copeland et al., “Adult Psychiatric Outcomes of Bullying and Being Bullied by Peers in Childhood and Adolescence,” JAMA Psychiatry 70 (2013): 419; S.

pages: 1,157 words: 379,558

Ashes to Ashes: America's Hundred-Year Cigarette War, the Public Health, and the Unabashed Triumph of Philip Morris by Richard Kluger


air freight, Albert Einstein, California gold rush, cognitive dissonance, corporate raider, desegregation, double entry bookkeeping, family office, feminist movement, full employment, ghettoisation, Indoor air pollution, medical malpractice, Mikhail Gorbachev, Plutocrats, plutocrats, publication bias, Ralph Nader, Ralph Waldo Emerson, RAND corporation, rent-seeking, risk tolerance, Ronald Reagan, selection bias, The Chicago School, the scientific method, Torches of Freedom, trade route, transaction costs, traveling salesman, union organizing, upwardly mobile, urban planning, urban renewal, War on Poverty

Public impressions to the contrary, no investigator had produced evidence remotely approaching in strength and consistency findings like those incriminating direct smoking by Wynder, Hammond and Horn, Doll and Hill, and Auerbach. The industry could thus retain the hope that a large-scale study might fail to show a correlation between lung cancer occurrence and exposure to ETS among nonsmokers. Such results, however, might not find their way into scientific journals because of a phenomenon known as “publication bias;” studies that produced negative results or did not report a statistically significant relationship were generally assigned a low priority among submissions. But in the spring of 1990, a Philip Morris scientist, Thomas J. Borelli, who bore the suggestive title of “manager of scientific issues,” was scouring about for unpublished studies on ETS and, while consulting the University Microfilms International Dissertation Information Service, struck gold.