Thomas Bayes

37 results back to index

pages: 561 words: 120,899

The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant From Two Centuries of Controversy by Sharon Bertsch McGrayne


Bayesian statistics, bioinformatics, British Empire, Claude Shannon: information theory, Daniel Kahneman / Amos Tversky, double helix, Edmond Halley, Fellow of the Royal Society, full text search, Henri Poincaré, Isaac Newton, John Markoff, John Nash: game theory, John von Neumann, linear programming, meta analysis, meta-analysis, Nate Silver, p-value, Pierre-Simon Laplace, placebo effect, prediction markets, RAND corporation, recommendation engine, Renaissance Technologies, Richard Feynman, Richard Feynman, Richard Feynman: Challenger O-ring, Ronald Reagan, speech recognition, statistical model, stochastic process, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, traveling salesman, Turing machine, Turing test, uranium enrichment, Yom Kippur War

A letter from the late Reverend Mr. Thomas Bayes, F.R.S., to John Canton, M.A. and F.R.S. Author(s): Mr. Bayes and Mr. Price. Philosophical Transactions (1683–1775) (53) 370–418. Royal Society. The original Bayes–Price article. Bebb, ED. (1935) Nonconformity and Social and Economic Life 1660–1800. London: Epworth Press. Bellhouse, David R. (2002) On some recently discovered manuscripts of Thomas Bayes. Historia Mathematica (29) 383–94. ———. (2007a) The Reverend Thomas Bayes, FRS: A biography to celebrate the tercentenary of his birth. Statistical Science (19:1) 3–43. With Dale (2003) the main source for Bayes’ life. ———. (2007b) Lord Stanhope’s papers on the Doctrine of Chances. Historia Mathematica (34) 173–86. Bru, Bernard. (1987) Preface in Thomas Bayes. Essai en vue de résoudre un problème de la doctrine des chances, trans. and ed., J-P Cléro.

Cone, Carl B. (1952) Torchbearer of Freedom: The Influence of Richard Price on Eighteenth-Century Thought. University of Kentucky Press. Dale, Andrew I. (1988) On Bayes’ theorem and the inverse Bernoulli theorem. Historia Mathematica (15) 348–60. ———. (1991) Thomas Bayes’s work on infinite series. Historia Mathematica (18) 312–27. ———. (1999) A History of Inverse Probability from Thomas Bayes to Karl Pearson. 2d ed. Springer. One of the foundational works in the history of probability. ———. (2003) Most Honourable Remembrance: The Life and Work of Thomas Bayes. Springer. With Bellhouse, the main source for Bayes’ life. Daston, Lorraine. (1988) Classical Probability in the Enlightenment. Princeton University Press. Deming WE, ed. (1940) Facsimiles of Two Papers by Bayes, With Commentaries by W. E.

Bayes was interred on April 15, which is often called the date of his death. The degraded condition of his vault may have contributed to the confusion. Second, the often-reproduced portrait of Thomas Bayes is almost assuredly of someone else named “T. Bayes.” The sketch first appeared in 1936 in History of Life Insurance in its Formative Years by Terence O’Donnell. However, the picture’s caption on page 335 says it is of “Rev. T. Bayes, Improver of the Columnar Method developed by Barrett,” and Barrett did not develop his method until 1810, a half-century after the death of “our” Rev. Thomas Bayes. Bellhouse (2004) first noticed that the portrait’s hairstyle is anachronistic. Sharon North, curator of Textiles and Fashion at the Victoria and Albert Museum, London, agrees: “The hairstyle in this portrait looks very 20th century. . . .

pages: 52 words: 16,113

The Laws of Medicine: Field Notes From an Uncertain Science by Siddhartha Mukherjee


Atul Gawande, cognitive dissonance, medical residency, randomized controlled trial, retrograde motion, stem cell, Thomas Bayes

Thank you for downloading this TED Books eBook. * * * Join our mailing list and get updates on new releases, deals, bonus content and other great books from TED Books and Simon & Schuster. CLICK HERE TO SIGN UP or visit us online to sign up at To Thomas Bayes (1702–1761), who saw uncertainty with such certainty “Are you planning to follow a career in Magical Laws, Miss Granger?” asked Scrimgeour. “No, I’m not,” retorted Hermione. “I’m hoping to do some good in the world!” J. K. Rowling The learned men of former ages employed a great part of their time and thoughts searching out the hidden causes of distemper, were curious in imagining the secret workmanship of nature and . . . putting all these fancies together, fashioned to themselves systems and hypotheses [that] diverted their enquiries from the true and advantageous knowledge of things.

It applies not only to medicine but to any other discipline that is predicated on predictions: economics or banking, gambling or astrology. The core logic holds true whether you are trying to forecast tomorrow’s weather or seeking to predict rises and falls in the stock market. It is a universal feature of all tests. .... The man responsible for this strange and illuminating idea was neither a doctor nor a scientist by trade. Born in Hertfordshire in 1702, Thomas Bayes was a clergyman and philosopher who served as the minister at the chapel in Tunbridge Wells, near London. He published only two significant papers in his lifetime—the first, a defense of God, and the second, a defense of Newton’s theory of calculus (it was a sign of the times that in 1732, a clergyman found no cognitive dissonance between these two efforts). His best-known work—on probability theory—was not published during his lifetime and was only rediscovered decades after his death.

pages: 266 words: 86,324

The Drunkard's Walk: How Randomness Rules Our Lives by Leonard Mlodinow


Albert Einstein, Alfred Russel Wallace, Antoine Gombaud: Chevalier de Méré, Atul Gawande, Brownian motion, butterfly effect, correlation coefficient, Daniel Kahneman / Amos Tversky, Donald Trump, feminist movement, forensic accounting, Gerolamo Cardano, Henri Poincaré, index fund, Isaac Newton, law of one price, pattern recognition, Paul Erdős, probability theory / Blaise Pascal / Pierre de Fermat, RAND corporation, random walk, Richard Feynman, Richard Feynman, Ronald Reagan, Stephen Hawking, Steve Jobs, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Bayes, V2 rocket, Watson beat the top human players on Jeopardy!

The experiment was still in progress, he reported, and now he was suing his former employer, who had produced a psychiatrist willing to testify that he suffered from paranoia. One of the paranoid delusions the former employer’s psychiatrist pointed to was the student’s alleged invention of a fictitious eighteenth-century minister. In particular, the psychiatrist scoffed at the student’s claim that this minister was an amateur mathematician who had created in his spare moments a bizarre theory of probability. The minister’s name, according to the student, was Thomas Bayes. His theory, the student asserted, described how to assess the chances that some event would occur if some other event also occurred. What are the chances that a particular student would be the subject of a vast secret conspiracy of experimental psychologists? Admittedly not huge. But what if one’s wife speaks one’s thoughts before one can utter them and co-workers foretell your professional fate over drinks in casual conversation?

And he presented the court with a mumbo jumbo of formulas and calculations regarding his hypothesis, concluding that the additional evidence meant that the probability was 999,999 in 1 million that he was right about the conspiracy. The enemy psychiatrist claimed that this mathematician-minister and his theory were figments of the student’s schizophrenic imagination. The student asked the professor to help him refute that claim. The professor agreed. He had good reason, for Thomas Bayes, born in London in 1701, really was a minister, with a parish at Tunbridge Wells. He died in 1761 and was buried in a park in London called Bunhill Fields, in the same grave as his father, Joshua, also a minister. And he indeed did invent a theory of “conditional probability” to show how the theory of probability can be extended from independent events to events whose outcomes are connected.

The professor supplied a deposition explaining Bayes’s existence and his theory, though not supporting the specific and dubious calculations that his former student claimed proved his sanity. The sad part of this story is not just the middle-aged schizophrenic himself, but the medical and legal team on the other side. It is unfortunate that some people suffer from schizophrenia, but even though drugs can help to mediate the illness, they cannot battle ignorance. And ignorance of the ideas of Thomas Bayes, as we shall see, resides at the heart of many serious mistakes in both medical diagnosis and legal judgment. It is an ignorance that is rarely addressed during a doctor’s or a lawyer’s professional training. We also make Bayesian judgments in our daily lives. A film tells the story of an attorney who has a great job, a charming wife, and a wonderful family. He loves his wife and daughter, but still he feels that something is missing in his life.

pages: 523 words: 143,139

Algorithms to Live By: The Computer Science of Human Decisions by Brian Christian, Tom Griffiths


4chan, Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, algorithmic trading, anthropic principle, asset allocation, autonomous vehicles, Bayesian statistics, Berlin Wall, Bill Duvall, bitcoin, Community Supported Agriculture, complexity theory, constrained optimization, cosmological principle, cryptocurrency, Danny Hillis, David Heinemeier Hansson, delayed gratification, dematerialisation, diversification, Donald Knuth, double helix, Elon Musk, fault tolerance, Fellow of the Royal Society, Firefox, first-price auction, Flash crash, Frederick Winslow Taylor, George Akerlof, global supply chain, Google Chrome, Henri Poincaré, information retrieval, Internet Archive, Jeff Bezos, John Nash: game theory, John von Neumann, knapsack problem, Lao Tzu, Leonard Kleinrock, linear programming, martingale, Nash equilibrium, natural language processing, NP-complete, P = NP, packet switching, Pierre-Simon Laplace, prediction markets, race to the bottom, RAND corporation, RFC: Request For Comment, Robert X Cringely, sealed-bid auction, second-price auction, self-driving car, Silicon Valley, Skype, sorting algorithm, spectrum auction, Steve Jobs, stochastic process, Thomas Bayes, Thomas Malthus, traveling salesman, Turing machine, urban planning, Vickrey auction, Vilfredo Pareto, Walter Mischel, Y Combinator, zero-sum game

The story begins in eighteenth-century England, in a domain of inquiry irresistible to great mathematical minds of the time, even those of the clergy: gambling. Reasoning Backward with the Reverend Bayes If we be, therefore, engaged by arguments to put trust in past experience, and make it the standard of our future judgement, these arguments must be probable only. —DAVID HUME More than 250 years ago, the question of making predictions from small data weighed heavily on the mind of the Reverend Thomas Bayes, a Presbyterian minister in the charming spa town of Tunbridge Wells, England. If we buy ten tickets for a new and unfamiliar raffle, Bayes imagined, and five of them win prizes, then it seems relatively easy to estimate the raffle’s chances of a win: 5/10, or 50%. But what if instead we buy a single ticket and it wins a prize? Do we really imagine the probability of winning to be 1/1, or 100%?

“The Unreasonable Effectiveness of Data”: The talk was derived from Halevy, Norvig, and Pereira, “The Unreasonable Effectiveness of Data.” “these arguments must be probable only”: An Enquiry Concerning Human Understanding, §IV, “Sceptical Doubts Concerning the Operations of the Understanding.” Bayes’s own history: Our brief biography draws on Dale, A History of Inverse Probability, and Bellhouse, “The Reverend Thomas Bayes.” in either 1746, ’47, ’48, or ’49: Bayes’s legendary paper, undated, had been filed between a pair of papers dated 1746 and 1749. See, e.g., McGrayne, The Theory That Would Not Die. defense of Newton’s newfangled “calculus”: An Introduction to the Doctrine of fluxions, and Defence of the Mathematicians against the Objections of the Author of the analyst, so far as they are assigned to affect their general methods of Reasoning.

Shedler. “An Anomaly in Space-Time Characteristics of Certain Programs Running in a Paging Machine.” Communications of the ACM 12, no. 6 (1969): 349–353. Belew, Richard K. Finding Out About: A Cognitive Perspective on Search Engine Technology and the WWW. Cambridge, UK: Cambridge University Press, 2000. Bell, Aubrey F. G. In Portugal. New York: John Lane, 1912. Bellhouse, David R. “The Reverend Thomas Bayes, FRS: A Biography to Celebrate the Tercentenary of His Birth.” Statistical Science 19 (2004): 3–43. Bellman, Richard. Dynamic Programming. Princeton, NJ: Princeton University Press, 1957. ______. “A Problem in the Sequential Design of Experiments.” Sankhyā: The Indian Journal of Statistics 16 (1956): 221–229. Bellows, Meghan L., and J. D. Luc Peterson. “Finding an Optimal Seating Chart.” Annals of Improbable Research (2012).

pages: 829 words: 186,976

The Signal and the Noise: Why So Many Predictions Fail-But Some Don't by Nate Silver


airport security, availability heuristic, Bayesian statistics, Benoit Mandelbrot, Berlin Wall, Bernie Madoff, big-box store, Black Swan, Broken windows theory, Carmen Reinhart, Claude Shannon: information theory, Climategate, Climatic Research Unit, cognitive dissonance, collapse of Lehman Brothers, collateralized debt obligation, complexity theory, computer age, correlation does not imply causation, Credit Default Swap, credit default swaps / collateralized debt obligations, cuban missile crisis, Daniel Kahneman / Amos Tversky, diversification, Donald Trump, Edmond Halley, Edward Lorenz: Chaos theory,, equity premium, Eugene Fama: efficient market hypothesis, everywhere but in the productivity statistics, fear of failure, Fellow of the Royal Society, Freestyle chess, fudge factor, George Akerlof, haute cuisine, Henri Poincaré, high batting average, housing crisis, income per capita, index fund, Intergovernmental Panel on Climate Change (IPCC), Internet Archive, invention of the printing press, invisible hand, Isaac Newton, James Watt: steam engine, John Nash: game theory, John von Neumann, Kenneth Rogoff, knowledge economy, locking in a profit, Loma Prieta earthquake, market bubble, Mikhail Gorbachev, Moneyball by Michael Lewis explains big data, Monroe Doctrine, mortgage debt, Nate Silver, negative equity, new economy, Norbert Wiener, PageRank, pattern recognition,, Pierre-Simon Laplace, prediction markets, Productivity paradox, random walk, Richard Thaler, Robert Shiller, Robert Shiller, Rodney Brooks, Ronald Reagan, Saturday Night Live, savings glut, security theater, short selling, Skype, statistical model, Steven Pinker, The Great Moderation, The Market for Lemons, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, too big to fail, transaction costs, transfer pricing, University of East Anglia, Watson beat the top human players on Jeopardy!, wikimedia commons

Finding patterns is easy in any kind of data-rich environment; that’s what mediocre gamblers do. The key is in determining whether the patterns represent noise or signal. But although there isn’t any one particular key to why Voulgaris might or might not bet on a given game, there is a particular type of thought process that helps govern his decisions. It is called Bayesian reasoning. The Improbable Legacy of Thomas Bayes Thomas Bayes was an English minister who was probably born in 1701—although it may have been 1702. Very little is certain about Bayes’s life, even though he lent his name to an entire branch of statistics and perhaps its most famous theorem. It is not even clear that anybody knows what Bayes looked like; the portrait of him that is commonly used in encyclopedia articles may have been misattributed.19 What is in relatively little dispute is that Bayes was born into a wealthy family, possibly in the southeastern English county of Hertfordshire.

On average, a team will go either over or under the total five games in a row about five times per season. That works out to 150 such streaks per season between the thirty NBA teams combined. 19. D. R. Bellhouse, “The Reverend Thomas Bayes FRS: A Biography to Celebrate the Tercentenary of His Birth,” Statistical Science, 19, 1, pp. 3–43; 2004. 20. Bayes may also have been an Arian, meaning someone who followed the teachings of the early Christian leader Arias and who regarded Jesus Christ as the divine son of God rather than (as most Christians then and now believe) a direct manifestation of God. 21. Thomas Bayes, “Divine Benevolence: Or an Attempt to Prove That the Principal End of the Divine Providence and Government Is the Happiness of His Creatures.” 22.

There are many reasons for it—some having to do with our psychological biases, some having to do with common methodological errors, and some having to do with misaligned incentives. Close to the root of the problem, however, is a flawed type of statistical thinking that these researchers are applying. FIGURE 8-6: A GRAPHICAL REPRESENTATION OF FALSE POSITIVES When Statistics Backtracked from Bayes Perhaps the chief intellectual rival to Thomas Bayes—although he was born in 1890, almost 120 years after Bayes’s death—was an English statistician and biologist named Ronald Aylmer (R. A.) Fisher. Fisher was a much more colorful character than Bayes, almost in the English intellectual tradition of Christopher Hitchens. He was handsome but a slovenly dresser,42 always smoking his pipe or his cigarettes, constantly picking fights with his real and imagined rivals.

pages: 589 words: 69,193

Mastering Pandas by Femi Anthony


Amazon Web Services, Bayesian statistics, correlation coefficient, correlation does not imply causation, Debian,, Internet of things, natural language processing, p-value, random walk, side project, statistical model, Thomas Bayes

The various topics that will be discussed are as follows: Introduction to Bayesian statistics Mathematical framework for Bayesian statistics Probability distributions Bayesian versus Frequentist statistics Introduction to PyMC and Monte Carlo simulation Illustration of Bayesian inference – Switchpoint detection Introduction to Bayesian statistics The field of Bayesian statistics is built on the work of Reverend Thomas Bayes, an 18th century statistician, philosopher, and Presbyterian minister. His famous Bayes' theorem, which forms the theoretical underpinnings for Bayesian statistics, was published posthumously in 1763 as a solution to the problem of inverse probability. For more details on this topic, refer to Inverse probability problems were all the rage in the early 18th century and were often formulated as follows: Suppose you play a game with a friend. There are 10 green balls and 7 red balls in bag 1 and 4 green and 7 red balls in bag 2.

pages: 415 words: 125,089

Against the Gods: The Remarkable Story of Risk by Peter L. Bernstein


Albert Einstein, Alvin Roth, Andrew Wiles, Antoine Gombaud: Chevalier de Méré, Bayesian statistics, Big bang: deregulation of the City of London, Bretton Woods, buttonwood tree, capital asset pricing model, cognitive dissonance, computerized trading, Daniel Kahneman / Amos Tversky, diversified portfolio, double entry bookkeeping, Edmond Halley, Edward Lloyd's coffeehouse, endowment effect, experimental economics, fear of failure, Fellow of the Royal Society, Fermat's Last Theorem, financial deregulation, financial innovation, full employment, index fund, invention of movable type, Isaac Newton, John Nash: game theory, John von Neumann, Kenneth Arrow, linear programming, loss aversion, Louis Bachelier, mental accounting, moral hazard, Myron Scholes, Nash equilibrium, Paul Samuelson, Philip Mirowski, probability theory / Blaise Pascal / Pierre de Fermat, random walk, Richard Thaler, Robert Shiller, Robert Shiller, spectrum auction, statistical model, The Bell Curve by Richard Herrnstein and Charles Murray, The Wealth of Nations by Adam Smith, Thomas Bayes, trade route, transaction costs, tulip mania, Vanguard fund, zero-sum game

With that innocent-sounding assertion, Bernoulli explained why King Midas was an unhappy man, why people tend to be risk-averse, and why prices must fall if customers are to be persuaded to buy more. Bernoulli's statement stood as the dominant paradigm of rational behavior for the next 250 years and laid the groundwork for modern principles of investment management. Almost exactly one hundred years after the collaboration between Pascal and Fermat, a dissident English minister named Thomas Bayes made a striking advance in statistics by demonstrating how to make better-informed decisions by mathematically blending new information into old information. Bayes's theorem focuses on the frequent occasions when we have sound intuitive judgments about the probability of some event and want to understand how to alter those judgments as actual events unfold. All the tools we use today in risk management and in the analysis of decisions and choice, from the strict rationality of game theory to the challenges of chaos theory, stem from the developments that took place between 1654 and 1760, with only two exceptions: In 1875, Francis Galton, an amateur mathematician who was Charles Darwin's first cousin, discovered regression to the mean, which explains why pride goeth before a fall and why clouds tend to have silver linings.

In this scenario, the data are given-10 pins, 12 pins, 1 pin-and the probability is the unknown. Questions put in this manner form the subject matter of what is known as inverse probability: with 12 defective pins out of 100,000, what is the probability that the true average ratio of defectives to the total is 0.01%? One of the most effective treatments of such questions was proposed by a minister named Thomas Bayes, who was born in 1701 and lived in Kent." Bayes was a Nonconformist; he rejected most of the ceremonial rituals that the Church of England had retained from the Catholic Church after their separation in the time of Henry VIII. Not much is known about Bayes, even though he was a Fellow of the Royal Society. One otherwise dry and impersonal textbook in statistics went so far as to characterize him as "enigmatic."16 He published nothing in mathematics while he was alive and left only two works that were published after his death but received little attention when they appeared.

The most exciting feature of all the achievements mentioned in this chapter is the daring idea that uncertainty can be measured. Uncertainty means unknown probabilities; to reverse Hacking's description of certainty, we can say that something is uncertain when our information is correct and an event fails to happen, or when our information is incorrect and an event does happen. Jacob Bernoulli, Abraham de Moivre, and Thomas Bayes showed how to infer previously unknown probabilities from the empirical facts of reality. These accomplishments are impressive for the sheer mental agility demanded, and audacious for their bold attack on the unknown. When de Moivre invoked ORIGINAL DESIGN, he made no secret of his wonderment at his own accomplishments. He liked to turn such phrases; at another point, he writes, "If we blind not ourselves with metaphysical dust we shall be led by a short and obvious way, to the acknowledgment of the great MAKER and GOUVERNOUR of all."25 We are by now well into the eighteenth century, when the Enlightenment identified the search for knowledge as the highest form of human activity.

pages: 411 words: 108,119

The Irrational Economist: Making Decisions in a Dangerous World by Erwann Michel-Kerjan, Paul Slovic


Andrei Shleifer, availability heuristic, bank run, Black Swan, Cass Sunstein, clean water, cognitive dissonance, collateralized debt obligation, complexity theory, conceptual framework, corporate social responsibility, Credit Default Swap, credit default swaps / collateralized debt obligations, cross-subsidies, Daniel Kahneman / Amos Tversky, endowment effect, experimental economics, financial innovation, Fractional reserve banking, George Akerlof, hindsight bias, incomplete markets, information asymmetry, Intergovernmental Panel on Climate Change (IPCC), invisible hand, Isaac Newton, iterative process, Kenneth Arrow, Loma Prieta earthquake, London Interbank Offered Rate, market bubble, market clearing, money market fund, moral hazard, mortgage debt, Pareto efficiency, Paul Samuelson, placebo effect, price discrimination, price stability, RAND corporation, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Reagan, source of truth, statistical model, stochastic process, The Wealth of Nations by Adam Smith, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, too big to fail, transaction costs, ultimatum game, University of East Anglia, urban planning, Vilfredo Pareto

This chapter explores a two-part conjecture: (1) After the occurrence of a virgin risk, people will overestimate the probability of another occurrence in the near future; (2) by contrast, after an experienced risk occurs, people will under-update their assessment of another event occurring soon. THE INABILITY TO USE BAYESIAN UPDATING IN EVERYDAY PRACTICE Risks are often posited to have an unknown true probability. The textbook model for how to proceed employs Bayes’ Rule (after eighteenth-century British mathematician Thomas Bayes), which shows mathematically how people should rationally change their existing beliefs about something in light of new evidence. Individuals use information available beforehand to form a so-called prior belief about the probability that an event will occur in a given period. New evidence about the risk is captured in something called a likelihood function, which expresses how plausible the evidence is given each possible value of the probability.

American Enterprise Institute American International Group (AIG) American Psychiatric Association, homosexuality and Americans-in-London problem Amygdala(fig.) Anxiety Arrow, Ken Arthur Andersen Assets Asteroid and Comet Impact Hazards Group (NASA) Asteroid explosions, risk of At War with the Weather (Kunreuther and Michel-Kerjan) Attention deficit disorder Awareness, behavioral change and Bali Action Plan (2007) Bargaining games. See also Game Theory; Theory of Games; Ultimatum Games Batson, Daniel Bayes, Thomas Bayes’ Rule Bayesian updating Behavior acceptable awareness and collective Behavior (continued) decision making and descriptive models of individual learned managerial market motivating myopic neuroscience and rational social uncertainty/risk and Behavioral biases Behavioral data, linking(fig.) Behavioral explanations Behavioral research Behavioral science Beliefs Benefits concentrated extreme sharing uncertain Bhopal disaster Black Death Blair, Tony Bonds catastrophe municipal Bowman, Edward Brain emotional/rational parts of Brain activity unfair offers and(fig.)

pages: 502 words: 107,657

Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die by Eric Siegel


Albert Einstein, algorithmic trading, Amazon Mechanical Turk, Apple's 1984 Super Bowl advert, backtesting, Black Swan, book scanning, bounce rate, business intelligence, business process, call centre, commoditize, computer age, conceptual framework, correlation does not imply causation, crowdsourcing, dark matter, data is the new oil,, Erik Brynjolfsson, Everything should be made as simple as possible, experimental subject, Google Glasses, happiness index / gross national happiness, job satisfaction, Johann Wolfgang von Goethe, lifelogging, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, mass immigration, Moneyball by Michael Lewis explains big data, Nate Silver, natural language processing, Netflix Prize, Network effects, Norbert Wiener, personalized medicine, placebo effect, prediction markets, Ray Kurzweil, recommendation engine, risk-adjusted returns, Ronald Coase, Search for Extraterrestrial Intelligence, self-driving car, sentiment analysis, software as a service, speech recognition, statistical model, Steven Levy, text mining, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Turing test, Watson beat the top human players on Jeopardy!, X Prize, Yogi Berra, zero-sum game

To prepare for this battle, we armed PA with powerful weaponry. The predictions were generated from machine learning across 50 million learning cases, each depicting a micro-lesson from history of the form, “User Mary was shown ad A and she did click it” (a positive case) or “User John was shown ad B and he did not click it” (a negative case). The learning technology employed to pick the best ad for each user was a Naïve Bayes model. Reverend Thomas Bayes was an eighteenth-century mathematician, and the “Naïve” part means that we take a very smart man’s ideas and compromise them in a way that simplifies yet makes their application feasible, resulting in a practical method that’s often considered good enough at prediction, and scales to the task at hand. I went with this method for its relative simplicity, since in fact I needed to generate 291 such models, one for each ad.

Apple Mac Apple Siri Argonne National Laboratory Arizona Petrified Forest National Park Arizona State University artificial intelligence (AI) about Mechanical Turk mind-reading technology possibility of, the Watson computer and Asimov, Isaac astronomy AT&T Research BellKor Netflix Prize teams Australia Austria automobile insurance crashes, predicting credit scores and accidents driver inatentiveness, predicting fraud predictions for Averitt aviation incidents Aviva Insurance (UK) AWK computer language B backtesting. See also test data Baesens, Ben bagging (bootstrap aggregating) Bangladesh Barbie dolls Bayes, Thomas (Bayes Network) Beane, Billy Beano Beaux, Alex behavioral predictors Bella Pictures BellKor BellKor Netflix Prize teams Ben Gurion University (Israel) Bernstein, Peter Berra, Yogi Big Bang Theory, The Big Bang theory Big Brother BigChaos team “big data” movement billing errors, predicting black box trading Black Swan, The (Taleb) blogs and blogging anxiety, predicting from entries collective intelligence and data glut and content in LiveJournal mood prediction research via nature of Blue Cross Blue Shield of Tennessee BMW BNSF Railway board games, predictive play of Bohr, Niels book titles, testing Bowie, David brain activity, predicting Brandeis, Louis Brasil Telecom (Oi) breast cancer, predicting Brecht, Bertolt Breiman, Leo Brigham Young University British Broadcasting Corporation (BBC) Brobst, Stephen Brooks, Mel Brynjolfsson, Eric buildings, predicting fault in Bullard, Ben burglaries, predicting business rules, decision trees and buying behavior, predicting C Cage, Nicolas Canadian Automobile Association Canadian Tire car crashes and harm, predicting CareerBuilder Carlin, George Carlson, Gretchen Carnegie Mellon University CART decision trees Castagno, Davide causality cell phone industry consumer behavior and dropped calls, predicting GPS data and location predicting Telenor (Norway) CellTel (African telecom) Central Tables.

pages: 137 words: 36,231

Information: A Very Short Introduction by Luciano Floridi


agricultural Revolution, Albert Einstein, bioinformatics, carbon footprint, Claude Shannon: information theory, conceptual framework, double helix, Douglas Engelbart, Douglas Engelbart, George Akerlof, Gordon Gekko, industrial robot, information asymmetry, intangible asset, Internet of things, invention of writing, John Nash: game theory, John von Neumann, moral hazard, Nash equilibrium, Norbert Wiener, Pareto efficiency, phenotype, Pierre-Simon Laplace, prisoner's dilemma, RAND corporation, RFID, Thomas Bayes, Turing machine, Vilfredo Pareto

The question she is implicitly asking is: `what is the probability thatA (= the email was infected), given the fact that B (= the email was blocked by the antivirus and placed in the quarantine folder) when, on average, 2% of my emails are actually infected and my antivirus is successful 95% of the time?'. Jill has just identified a way of acquiring (learning) the missing piece of information that will help her to adopt the right strategy: if the chance that some emails in the quarantine folder might not be infected is very low, she will check it only occasionally. How could she obtain such a missing piece of information? The answer is by using a Bayesian approach. Thomas Bayes (1702-1761) was a Presbyterian minister and English mathematician whose investigations into probability, published posthumously, led to what is now known as Bayes' theorem and a new branch of applications of probability theory. The theorem calculates the posterior probability of an eventA given event B (that is, P(AIB) on the basis of the prior probability ofA (that is, P(A)). Basically, it tells us what sort of information can be retrodicted.

pages: 189 words: 57,632

Content: Selected Essays on Technology, Creativity, Copyright, and the Future of the Future by Cory Doctorow


book scanning, Brewster Kahle, Burning Man,, informal economy, information retrieval, Internet Archive, invention of movable type, Jeff Bezos, Law of Accelerating Returns, Metcalfe's law, moral panic, mutually assured destruction, new economy, optical character recognition, patent troll, pattern recognition, peer-to-peer, Ponzi scheme, post scarcity, QWERTY keyboard, Ray Kurzweil, RFID, Sand Hill Road, Skype, slashdot, social software, speech recognition, Steve Jobs, Thomas Bayes, Turing test, Vernor Vinge

The Future of Internet Immune Systems (Originally published on InformationWeek's Internet Evolution, November 19, 2007) Bunhill Cemetery is just down the road from my flat in London. It’s a handsome old boneyard, a former plague pit (“Bone hill” — as in, there are so many bones under there that the ground is actually kind of humped up into a hill). There are plenty of luminaries buried there — John “Pilgrim’s Progress” Bunyan, William Blake, Daniel Defoe, and assorted Cromwells. But my favorite tomb is that of Thomas Bayes, the 18th-century statistician for whom Bayesian filtering is named. Bayesian filtering is plenty useful. Here’s a simple example of how you might use a Bayesian filter. First, get a giant load of non-spam emails and feed them into a Bayesian program that counts how many times each word in their vocabulary appears, producing a statistical breakdown of the word-frequency in good emails. Then, point the filter at a giant load of spam (if you’re having a hard time getting a hold of one, I have plenty to spare), and count the words in it.

pages: 229 words: 67,599

The Logician and the Engineer: How George Boole and Claude Shannon Created the Information Age by Paul J. Nahin


Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, Any sufficiently advanced technology is indistinguishable from magic, Claude Shannon: information theory, conceptual framework, Edward Thorp, Fellow of the Royal Society, finite state, four colour theorem, Georg Cantor, Grace Hopper, Isaac Newton, John von Neumann, knapsack problem, New Journalism, Pierre-Simon Laplace, reversible computing, Richard Feynman, Richard Feynman, Schrödinger's Cat, Steve Jobs, Steve Wozniak, thinkpad, Thomas Bayes, Turing machine, Turing test, V2 rocket

The “general doctrine” does have a sort of plausibility to it: “if Y then not X” when “reversed” could be thought to imply “if X then not Y.” Boole argued that this is not so, using the ideas of the previous section, and showed that P( | X) is given by a considerably more involved expression than simply “p.” What Boole did was not really original, as conditional probability had been studied a century before by the English philosopher and minister Thomas Bayes (1701–1761), whose work was published posthumously in 1764 in the Philosophical Transactions of the Royal Society of London, where it was then promptly forgotten for twenty years until the great French mathematician Pierre-Simon Laplace (1749–1827) endorsed Bayes’s results. What Boole did, then, with the following analysis, was remind his readers what the Reverend Bayes had done a hundred years before.

Everydata: The Misinformation Hidden in the Little Data You Consume Every Day by John H. Johnson


Affordable Care Act / Obamacare, Black Swan, business intelligence, Carmen Reinhart, cognitive bias, correlation does not imply causation, Daniel Kahneman / Amos Tversky, Donald Trump,, Kenneth Rogoff, labor-force participation, lake wobegon effect, Long Term Capital Management, Mercator projection, Mercator projection distort size, especially Greenland and Africa, meta analysis, meta-analysis, Nate Silver, obamacare, p-value, PageRank, pattern recognition, publication bias, QR code, randomized controlled trial, risk-adjusted returns, Ronald Reagan, selection bias, statistical model, The Signal and the Noise by Nate Silver, Thomas Bayes, Tim Cook: Apple, wikimedia commons, Yogi Berra

Ellen Davis, “Committing the ‘Gambler’s Fallacy’ May Be in the Cards, New Research Shows,” Texas A&M Health Science Center website, March 9, 2015,­committing-​­the-​­gamblers-​­fallacy-​­may‑be‑in‑­the -​­cards-​­new-​­research-​­shows. Thanks to Ron Friedman for the find. 27. There’s another way of looking at this, known as Bayesian probability (after the ­eighteenth-​­century English mathematician Thomas Bayes). With Bayesian probability, you use the data gathered to update your initial beliefs after the fact. It’s the opposite of the way in which the gambler’s fallacy works. As one of John’s colleagues pointed out, it’s the difference between knowing that a coin is fair and learning about the coin. So, a Bayesian might flip a coin 10 times, get heads all 10 times, and adjust his probability to say that the coin was always more likely to land heads up.

pages: 269 words: 74,955

The Crash Detectives by Christine Negroni

Air France Flight 447, Airbus A320, Captain Sullenberger Hudson, Checklist Manifesto, computer age, crew resource management, crowdsourcing, low cost carrier, Richard Feynman, Richard Feynman, South China Sea, Tenerife airport disaster, Thomas Bayes, US Airways Flight 1549

Some of the credit for finally finding the submerged airliner goes to Metron Scientific Solutions, a company staffed with pencil-wielding mathematicians who used probability, logic, and numbers to conclude that the likely resting place of the plane was a narrow slice of ocean that had already been checked. “A lack of success tells you about where it is not, and that contributes to knowledge,” said Larry Stone, chief scientist at Metron. Talk about having a positive point of view. The Metron method is based on Bayesian probability, the theory of eighteenth-century statistician and philosopher Thomas Bayes, whose first published work, Divine Benevolence, was equally optimistic because it attempted to prove that God wants us to be happy. Using Bayesian logic to look for missing airplanes, as interpreted by Metron, involves taking all kinds of input about the missing thing (even conflicting input) and assigning levels of certainty or uncertainty to each. Everything gets a weight, and everything gets revised as things change.

The Singularity Is Near: When Humans Transcend Biology by Ray Kurzweil


additive manufacturing, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, anthropic principle, Any sufficiently advanced technology is indistinguishable from magic, artificial general intelligence, Asilomar, augmented reality, autonomous vehicles, Benoit Mandelbrot, Bill Joy: nanobots, bioinformatics, brain emulation, Brewster Kahle, Brownian motion, business intelligence,, call centre, carbon-based life, cellular automata, Claude Shannon: information theory, complexity theory, conceptual framework, Conway's Game of Life, cosmological constant, cosmological principle, cuban missile crisis, data acquisition, Dava Sobel, David Brooks, Dean Kamen, disintermediation, double helix, Douglas Hofstadter,, epigenetics, factory automation, friendly AI, George Gilder, Gödel, Escher, Bach, informal economy, information retrieval, invention of the telephone, invention of the telescope, invention of writing, Isaac Newton, iterative process, Jaron Lanier, Jeff Bezos, job automation, job satisfaction, John von Neumann, Kevin Kelly, Law of Accelerating Returns, life extension, lifelogging, linked data, Loebner Prize, Louis Pasteur, mandelbrot fractal, Mikhail Gorbachev, mouse model, Murray Gell-Mann, mutually assured destruction, natural language processing, Network effects, new economy, Norbert Wiener, oil shale / tar sands, optical character recognition, pattern recognition, phenotype, premature optimization, randomized controlled trial, Ray Kurzweil, remote working, reversible computing, Richard Feynman, Richard Feynman, Robert Metcalfe, Rodney Brooks, Search for Extraterrestrial Intelligence, selection bias, semantic web, Silicon Valley, Singularitarianism, speech recognition, statistical model, stem cell, Stephen Hawking, Stewart Brand, strong AI, superintelligent machines, technological singularity, Ted Kaczynski, telepresence, The Coming Technological Singularity, Thomas Bayes, transaction costs, Turing machine, Turing test, Vernor Vinge, Y2K, Yogi Berra

He plans to develop a system incorporating all human ideas.167 One application would be to inform policy makers of which ideas are held by which community. Bayesian Nets. Over the last decade a technique called Bayesian logic has created a robust mathematical foundation for combining thousands or even millions of such probabilistic rules in what are called "belief networks" or Bayesian nets. Originally devised by English mathematician Thomas Bayes and published posthumously in 1763, the approach is intended to determine the likelihood of future events based on similar occurrences in the past.168 Many expert systems based on Bayesian techniques gather data from experience in an ongoing fashion, thereby continually learning and improving their decision making. The most promising type of spam filters are based on this method. I personally use a spam filter called SpamBayes, which trains itself on e-mail that you have identified as either "spam" or "okay."169 You start out by presenting a folder of each to the filter.

Anthes, "Computerizing Common Sense," Computerworld, April 8, 2002,,11280,69881,00.html. 167. Kristen Philipkoski, "Now Here's a Really Big Idea," Wired News, November 25, 2002,,1282,56374,00.html, reporting on Darryl Macer, "The Next Challenge Is to Map the Human Mind," Nature 420 (November 14, 2002): 121; see also a description of the project at 168. Thomas Bayes, "An Essay Towards Solving a Problem in the Doctrine of Chances," published in 1763, two years after his death in 1761. 169. SpamBayes spam filter, 170. Lawrence R. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition," Proceedings of the IEEE 77 (1989): 257–86. For a mathematical treatment of Markov models, see 171.

pages: 350 words: 103,270

The Devil's Derivatives: The Untold Story of the Slick Traders and Hapless Regulators Who Almost Blew Up Wall Street . . . And Are Ready to Do It Again by Nicholas Dunbar


asset-backed security, bank run, banking crisis, Basel III, Black Swan, Black-Scholes formula, bonus culture, break the buck, capital asset pricing model, Carmen Reinhart, Cass Sunstein, collateralized debt obligation, commoditize, Credit Default Swap, credit default swaps / collateralized debt obligations, delayed gratification, diversification, Edmond Halley, facts on the ground, financial innovation, fixed income, George Akerlof, implied volatility, index fund, interest rate derivative, interest rate swap, Isaac Newton, John Meriwether, Kenneth Rogoff, Long Term Capital Management, margin call, market bubble, money market fund, Myron Scholes, Nick Leeson, Northern Rock, offshore financial centre, Paul Samuelson, price mechanism, regulatory arbitrage, rent-seeking, Richard Thaler, risk tolerance, risk/return, Ronald Reagan, shareholder value, short selling, statistical model, The Chicago School, Thomas Bayes, time value of money, too big to fail, transaction costs, value at risk, Vanguard fund, yield curve, zero-sum game

Note that we ignore any mention of time, or the time value of money, in this example, which is the equivalent of setting the risk-free interest rate to zero. 6. One might argue that since the market values the loans at $800 million, the bank ought to write down the value of the equity investment to zero. However, accounting rules for loan books don’t require such recognitions to take place. 7. After de Moivre’s death, the refinement of mortality calculations was continued in London by Richard Price, friend of Thomas Bayes and Benjamin Franklin, and founding actuary of the Equitable Life Assurance Society. 8. Arturo Cifuentes and Gerard O’Connor, “The Binomial Expansion Method Applied to CBO/CLO Analysis,” Moody’s Investors Service special report, December 13, 1996. 9. Ibid. 10. For a detailed account of the invention of BISTRO, see Gillian Tett, Fool’s Gold: How the Bold Dream of a Small Tribe at J.P. Morgan Was Corrupted by Wall Street Greed and Unleashed a Catastrophe (New York: Free Press, 2009). 11.

pages: 519 words: 102,669

Programming Collective Intelligence by Toby Segaran


always be closing, correlation coefficient, Debian,, Firefox, full text search, information retrieval, PageRank, prediction markets, recommendation engine, slashdot, Thomas Bayes, web application

In, create a subclass of classifier called naivebayes, and create a docprob method that extracts the features (words) and multiplies all their probabilities together to get an overall probability: class naivebayes(classifier): def docprob(self,item,cat): features=self.getfeatures(item) # Multiply the probabilities of all the features together p=1 for f in features: p*=self.weightedprob(f,cat,self.fprob) return p You now know how to calculate Pr(Document | Category), but this isn't very useful by itself. In order to classify documents, you really need Pr(Category | Document). In other words, given a specific document, what's the probability that it fits into this category? Fortunately, a British mathematician named Thomas Bayes figured out how to do this about 250 years ago. A Quick Introduction to Bayes' Theorem Bayes' Theorem is a way of flipping around conditional probabilities. It's usually written as: Pr(A | B) = Pr(B | A) × Pr(A)/Pr(B) In the example, this becomes: Pr(Category | Document) = Pr(Document | Category) × Pr(Category) / Pr(Document) The previous section showed how to calculate Pr(Document | Category), but what about the other two values in the equation?

pages: 339 words: 105,938

The Skeptical Economist: Revealing the Ethics Inside Economics by Jonathan Aldred


airport security, Berlin Wall, carbon footprint, citizen journalism, clean water, cognitive dissonance, congestion charging, correlation does not imply causation, Diane Coyle, endogenous growth, experimental subject, Fall of the Berlin Wall, first-past-the-post, framing effect, greed is good, happiness index / gross national happiness, Intergovernmental Panel on Climate Change (IPCC), invisible hand, job satisfaction, John Maynard Keynes: Economic Possibilities for our Grandchildren, labour market flexibility, laissez-faire capitalism, libertarian paternalism, new economy, Pareto efficiency, pension reform, positional goods, Ralph Waldo Emerson, RAND corporation, risk tolerance, school choice, spectrum auction, Thomas Bayes, trade liberalization, ultimatum game

This is a very broad question, so we shall focus on just one aspect of it, namely the practice of quantifying uncertainty and ignorance — inventing probabilities when there is no basis to do so. The practice is widespread among economists because many of them believe that, no matter how extreme the uncertainty, effective probabilities always exist. This view is termed ‘subjective Bayesianism’ (hereafter Bayesianism for short), from the Reverend Thomas Bayes, an 18th-century English mathematician.35 Its implications are startling. Bayesians believe there is no such thing as pure uncertainty in the sense I have defined it. They assert that we always use probabilities, consciously or otherwise, when outcomes are not certain. The issues are more clearly depicted in simple gambling games than messy real-world choices; the Ellsberg Paradox (see box opposite) is a classic illustration.

pages: 294 words: 81,292

Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat


3D printing, AI winter, Amazon Web Services, artificial general intelligence, Asilomar, Automated Insights, Bayesian statistics, Bernie Madoff, Bill Joy: nanobots, brain emulation, cellular automata, Chuck Templeton: OpenTable, cloud computing, cognitive bias, commoditize, computer vision, cuban missile crisis, Daniel Kahneman / Amos Tversky, Danny Hillis, data acquisition, don't be evil, drone strike, Extropian, finite state, Flash crash, friendly AI, friendly fire, Google Glasses, Google X / Alphabet X, Isaac Newton, Jaron Lanier, John Markoff, John von Neumann, Kevin Kelly, Law of Accelerating Returns, life extension, Loebner Prize, lone genius, mutually assured destruction, natural language processing, Nicholas Carr, optical character recognition, PageRank, pattern recognition, Peter Thiel, prisoner's dilemma, Ray Kurzweil, Rodney Brooks, Search for Extraterrestrial Intelligence, self-driving car, semantic web, Silicon Valley, Singularitarianism, Skype, smart grid, speech recognition, statistical model, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Steve Wozniak, strong AI, Stuxnet, superintelligent machines, technological singularity, The Coming Technological Singularity, Thomas Bayes, traveling salesman, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, zero day

But by the time the tragedy unfolded, Holtzman told me, Good had retired. He was not in his office but at home, perhaps calculating the probability of God’s existence. According to Dr. Holtzman, sometime before he died, Good updated that probability from zero to point one. He did this because as a statistician, he was a long-term Bayesian. Named for the eighteenth-century mathematician and minister Thomas Bayes, Bayesian statistics’ main idea is that in calculating the probability of some statement, you can start with a personal belief. Then you update that belief as new evidence comes in that supports your statement or doesn’t. If Good’s original disbelief in God had remained 100 percent, no amount of data, not even God’s appearance, could change his mind. So, to be consistent with his Bayesian perspective, Good assigned a small positive probability to the existence of God to make sure he could learn from new data, if it arose.

pages: 317 words: 100,414

Superforecasting: The Art and Science of Prediction by Philip Tetlock, Dan Gardner


Affordable Care Act / Obamacare, Any sufficiently advanced technology is indistinguishable from magic, availability heuristic, Black Swan, butterfly effect, cloud computing, cuban missile crisis, Daniel Kahneman / Amos Tversky, desegregation, drone strike, Edward Lorenz: Chaos theory, forward guidance, Freestyle chess, fundamental attribution error, germ theory of disease, hindsight bias, index fund, Jane Jacobs, Jeff Bezos, Kenneth Arrow, Mikhail Gorbachev, Mohammed Bouazizi, Nash equilibrium, Nate Silver, obamacare, pattern recognition, performance metric, Pierre-Simon Laplace, place-making, placebo effect, prediction markets, quantitative easing, random walk, randomized controlled trial, Richard Feynman, Richard Feynman, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Reagan, Saturday Night Live, Silicon Valley, Skype, statistical model, stem cell, Steve Ballmer, Steve Jobs, Steven Pinker, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, Thomas Bayes, Watson beat the top human players on Jeopardy!

If he says, “It’s to the left,” the likelihood of the first ball being on the right side of the table increases a little more. Keep repeating the process and you slowly narrow the range of the possible locations, zeroing in on the truth—although you will never eliminate uncertainty entirely.16 If you’ve taken Statistics 101, you may recall a version of this thought experiment was dreamt up by Thomas Bayes. A Presbyterian minister, educated in logic, Bayes was born in 1701, so he lived at the dawn of modern probability theory, a subject to which he contributed with “An Essay Towards Solving a Problem in the Doctrine of Chances.” That essay, in combination with the work of Bayes’ friend Richard Price, who published Bayes’ essay posthumously in 1761, and the insights of the great French mathematician Pierre-Simon Laplace, ultimately produced Bayes’ theorem.

pages: 370 words: 94,968

The Most Human Human: What Talking With Computers Teaches Us About What It Means to Be Alive by Brian Christian


4chan, Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Bertrand Russell: In Praise of Idleness, carbon footprint, cellular automata, Claude Shannon: information theory, cognitive dissonance, commoditize, complexity theory, crowdsourcing, David Heinemeier Hansson, Donald Trump, Douglas Hofstadter, George Akerlof, Gödel, Escher, Bach, high net worth, Isaac Newton, Jacques de Vaucanson, Jaron Lanier, job automation, l'esprit de l'escalier, Loebner Prize, Menlo Park, Ray Kurzweil, RFID, Richard Feynman, Richard Feynman, Ronald Reagan, Skype, statistical model, Stephen Hawking, Steve Jobs, Steven Pinker, theory of mind, Thomas Bayes, Turing machine, Turing test, Von Neumann architecture, Watson beat the top human players on Jeopardy!, zero-sum game

I walk out of the Brighton Centre, to the bracing sea air for a minute, and into a small, locally owned shoe store looking for a gift to bring back home to my girlfriend; the shopkeeper notices my accent; I tell her I’m from Seattle; she is a grunge fan; I comment on the music playing in the store; she says it’s Florence + the Machine; I tell her I like it and that she would probably like Feist … I walk into a tea and scone store called the Mock Turtle and order the British equivalent of coffee and a donut, except it comes with thirteen pieces of silverware and nine pieces of flatware; I am so in England, I think; an old man, probably in his eighties, is shakily eating a pastry the likes of which I’ve never seen; I ask him what it is; “coffee meringue,” he says and remarks on my accent; an hour later he is telling me about World War II, the exponentially increasing racial diversity of Britain, that House of Cards is a pretty accurate depiction of British politics, minus the murders, but that really I should watch Spooks; do you get Spooks on cable, he is asking me … I meet my old boss for dinner; and after a couple years of being his research assistant and occasionally co-author, and after a brief thought of becoming one of his Ph.D. students, after a year of our paths not really crossing, we negotiate whether our formerly collegial and hierarchical relationship, now that its context is removed, simply dries up or flourishes into a domain-general friendship; we are ordering appetizers and saying something about Wikipedia, something about Thomas Bayes, something about vegetarian dining … Laurels are of no use. If you de-anonymized yourself in the past, great. But that was that. And now, you begin again. 1. These logs would, three years later, be put on the IBM website, albeit in incomplete form and with so little fanfare that Kasparov himself wouldn’t find out about them until 2005. Epilogue: The Unsung Beauty of the Glassware Cabinet The Most Room-Like Room: The Cornell Box The image-processing world, it turns out, has a close analogue to the Turing test, called “the Cornell box,” which is a small model of a room with one red wall and one green wall (the others are white) and two blocks sitting inside it.

pages: 336 words: 113,519

The Undoing Project: A Friendship That Changed Our Minds by Michael Lewis


Albert Einstein, availability heuristic, Cass Sunstein, choice architecture, complexity theory, Daniel Kahneman / Amos Tversky, Donald Trump, Douglas Hofstadter, endowment effect, feminist movement, framing effect, hindsight bias, John von Neumann, Kenneth Arrow, loss aversion, medical residency, Menlo Park, Murray Gell-Mann, Nate Silver, New Journalism, Paul Samuelson, Richard Thaler, Saturday Night Live, statistical model, the new new thing, Thomas Bayes, Walter Mischel, Yom Kippur War

The subject picked one of the bags at random and, without glancing inside the bag, began to pull chips out of it, one at a time. After extracting each chip, he’d give the psychologists his best guess of the odds that the bag he was holding was filled with mostly red, or mostly white, chips. The beauty of the experiment was that there was a correct answer to the question: What is the probability that I am holding the bag of mostly red chips? It was provided by a statistical formula called Bayes’s theorem (after Thomas Bayes, who, strangely, left the formula for others to discover in his papers after his death, in 1761). Bayes’s rule allowed you to calculate the true odds, after each new chip was pulled from it, that the book bag in question was the one with majority white, or majority red, chips. Before any chips had been withdrawn, those odds were 50:50—the bag in your hands was equally likely to be either majority red or majority white.

Gaming the Vote: Why Elections Aren't Fair (And What We Can Do About It) by William Poundstone

affirmative action, Albert Einstein, Debian, desegregation, Donald Trump,, Everything should be made as simple as possible, global village, guest worker program, hiring and firing, illegal immigration, invisible hand, jimmy wales, John Nash: game theory, John von Neumann, Kenneth Arrow, manufacturing employment, Nash equilibrium, Paul Samuelson, Pierre-Simon Laplace, prisoner's dilemma, Ralph Nader, RAND corporation, Ronald Reagan, Silicon Valley, slashdot, the map is not the territory, Thomas Bayes, transcontinental railway, Unsafe at Any Speed, Y2K

Smith holds or has applied for patents covering such exotica as a computer made out of DNA, theft-proof credit cards, a 3-D vision process, and a magnetic catapult that could be used for launching satellites, In December 2000, with the Supreme Court deciding a bitterly contested presidency, Smith completed an article purporting to demonstrate the superiority of a system that no one had taken seriously, range voting. He began with an idea for comparing the merits of different voting systems, using a measure called Bayesian regret. The "Bayes" part refers to eighteenth-century English mathematician Thomas Bayes, a pioneer of probability theory, "Bayesian regret" is a statistical term that Smith defines as "expected avoidable human unhappiness." In other words, Smith tried to gauge how voting systems fail the voters by electing candidates other than the one who would have resulted in the greatest overall satisfaction, To do this, he ran a large series of computer simulations of elections. In each of his simulations, virtual voters were assigned utilities (degrees of happiness, measured numerically) for simulated candidates.

pages: 398 words: 120,801

Little Brother by Cory Doctorow


airport security, Bayesian statistics, Berlin Wall, citizen journalism, Firefox, game design, Golden Gate Park, Haight Ashbury, Internet Archive, Isaac Newton, Jane Jacobs, Jeff Bezos, mail merge, RFID, Sand Hill Road, Silicon Valley, slashdot, Steve Jobs, Steve Wozniak, Thomas Bayes, web of trust, zero day

No one could tell which of the Internet's packets were Xnet and which ones were just plain old banking and e-commerce and other encrypted communication. You couldn't find out who was tying the Xnet, let alone who was using the Xnet. But what about Dad's "Bayesian statistics?" I'd played with Bayesian math before. Darryl and I once tried to write our own better spam filter and when you filter spam, you need Bayesian math. Thomas Bayes was an 18th century British mathematician that no one cared about until a couple hundred years after he died, when computer scientists realized that his technique for statistically analyzing mountains of data would be super-useful for the modern world's info-Himalayas. Here's some of how Bayesian stats work. Say you've got a bunch of spam. You take every word that's in the spam and count how many times it appears.

pages: 755 words: 121,290

Statistics hacks by Bruce Frey


Bayesian statistics, Berlin Wall, correlation coefficient, Daniel Kahneman / Amos Tversky, distributed generation,, feminist movement, game design, Hacker Ethic, index card, Milgram experiment, p-value, place-making, RFID, Search for Extraterrestrial Intelligence, SETI@home, Silicon Valley, statistical model, Thomas Bayes

What about the accuracy of a negative result? Of the 9,102 women who will score negative on the screening, 12 actually have cancer. This is a relatively small 1/10 of 1 percent, but the testing will miss those people altogether, and they will not receive treatment. Why It Works Medical screening accuracy uses a specific application of a generalized approach to conditional probability attributed to Thomas Bayes, a philosopher and mathematician in the 1700s. "If this, then what are the chances that..." is a conditional probability question. Bayes's approach to conditional probabilities was to look at the naturally occurring frequencies of events. The basic formula for estimating the chance that one has a disease if one has a positive test result is: Expressed as conditional probabilities, the formula is: To answer the all-important question in our breast cancer example ("If a woman scores a positive test result, how likely is she to have breast cancer?")

pages: 483 words: 141,836

Red-Blooded Risk: The Secret History of Wall Street by Aaron Brown, Eric Kim


activist fund / activist shareholder / activist investor, Albert Einstein, algorithmic trading, Asian financial crisis, Atul Gawande, backtesting, Basel III, Bayesian statistics, beat the dealer, Benoit Mandelbrot, Bernie Madoff, Black Swan, capital asset pricing model, central bank independence, Checklist Manifesto, corporate governance, creative destruction, credit crunch, Credit Default Swap, disintermediation, distributed generation, diversification, diversified portfolio, Edward Thorp, Emanuel Derman, Eugene Fama: efficient market hypothesis, experimental subject, financial innovation, illegal immigration, implied volatility, index fund, Long Term Capital Management, loss aversion, margin call, market clearing, market fundamentalism, market microstructure, money market fund, money: store of value / unit of account / medium of exchange, moral hazard, Myron Scholes, natural language processing, open economy, Pierre-Simon Laplace, pre–internet, quantitative trading / quantitative finance, random walk, Richard Thaler, risk tolerance, risk-adjusted returns, risk/return, road to serfdom, Robert Shiller, Robert Shiller, shareholder value, Sharpe ratio, special drawing rights, statistical arbitrage, stochastic volatility, The Myth of the Rational Market, Thomas Bayes, too big to fail, transaction costs, value at risk, yield curve

Second, we know nothing about the accuracy of this statement in particular; we only make a claim about the long-term accuracy of lots of statements. This is how we turn an event that has already happened—drawing nine red marbles out of 10—into a hypothetical coin-flip gambling game that can be repeated indefinitely. The main alternative to frequentist statistics today is the Bayesian view. It is named for Thomas Bayes, an eighteenth-century theorist, but it was Pierre-Simon Laplace who put forth the basic ideas. It was not until the twentieth century, however, that researchers, including Richard Cox and Bruno de Finetti, created the modern formulation. In the Bayesian view of the urn, you must have some prior belief about the number of red marbles in the urn. For example, you might believe that any number from 0 to 100 red marbles is equally likely.

pages: 471 words: 124,585

The Ascent of Money: A Financial History of the World by Niall Ferguson


Admiral Zheng, Andrei Shleifer, Asian financial crisis, asset allocation, asset-backed security, Atahualpa, bank run, banking crisis, banks create money, Black Swan, Black-Scholes formula, Bonfire of the Vanities, Bretton Woods, BRICs, British Empire, capital asset pricing model, capital controls, Carmen Reinhart, Cass Sunstein, central bank independence, collateralized debt obligation, colonial exploitation, commoditize, Corn Laws, corporate governance, creative destruction, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, currency manipulation / currency intervention, currency peg, Daniel Kahneman / Amos Tversky, deglobalization, diversification, diversified portfolio, double entry bookkeeping, Edmond Halley, Edward Glaeser, Edward Lloyd's coffeehouse, financial innovation, financial intermediation, fixed income, floating exchange rates, Fractional reserve banking, Francisco Pizarro, full employment, German hyperinflation, Hernando de Soto, high net worth, hindsight bias, Home mortgage interest deduction, Hyman Minsky, income inequality, information asymmetry, interest rate swap, Intergovernmental Panel on Climate Change (IPCC), Isaac Newton, iterative process, John Meriwether, joint-stock company, joint-stock limited liability company, Joseph Schumpeter, Kenneth Arrow, Kenneth Rogoff, knowledge economy, labour mobility, Landlord’s Game, liberal capitalism, London Interbank Offered Rate, Long Term Capital Management, market bubble, market fundamentalism, means of production, Mikhail Gorbachev, money market fund, money: store of value / unit of account / medium of exchange, moral hazard, mortgage debt, mortgage tax deduction, Myron Scholes, Naomi Klein, negative equity, Nick Leeson, Northern Rock, Parag Khanna, pension reform, price anchoring, price stability, principal–agent problem, probability theory / Blaise Pascal / Pierre de Fermat, profit motive, quantitative hedge fund, RAND corporation, random walk, rent control, rent-seeking, reserve currency, Richard Thaler, Robert Shiller, Robert Shiller, Ronald Reagan, savings glut, seigniorage, short selling, Silicon Valley, South Sea Bubble, sovereign wealth fund, spice trade, structural adjustment programs, technology bubble, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, Thomas Bayes, Thomas Malthus, Thorstein Veblen, too big to fail, transaction costs, value at risk, Washington Consensus, Yom Kippur War

In 1738 the Swiss mathematician Daniel Bernoulli proposed that ‘The value of an item must not be based on its price, but rather on the utility that it yields’, and that the ‘utility resulting from any small increase in wealth will be inversely proportionate to the quantity of goods previously possessed’ - in other words $100 is worth more to someone on the median income than to a hedge fund manager. 6. Inference. In his ‘Essay Towards Solving a Problem in the Doctrine of Chances’ (published posthumously in 1764), Thomas Bayes set himself the following problem: ‘Given the number of times in which an unknown event has happened and failed; Required the chance that the probability of its happening in a single trial lies somewhere between any two degrees of probability that can be named.’ His resolution of the problem - ‘The probability of any event is the ratio between the value at which an expectation depending on the happening of the event ought to be computed, and the chance of the thing expected upon it’s [sic] happening’ - anticipates the modern formulation that expected utility is the probability of an event times the payoff received in case of that event.18 In short, it was not merchants but mathematicians who were the true progenitors of modern insurance.

pages: 320 words: 33,385

Market Risk Analysis, Quantitative Methods in Finance by Carol Alexander


asset allocation, backtesting, barriers to entry, Brownian motion, capital asset pricing model, constrained optimization, credit crunch, Credit Default Swap, discounted cash flows, discrete time, diversification, diversified portfolio,, fixed income, implied volatility, interest rate swap, market friction, market microstructure, p-value, performance metric, quantitative trading / quantitative finance, random walk, risk tolerance, risk-adjusted returns, risk/return, Sharpe ratio, statistical arbitrage, statistical model, stochastic process, stochastic volatility, Thomas Bayes, transaction costs, value at risk, volatility smile, Wiener process, yield curve, zero-sum game

For instance, if we threw a fair die 600 times we would expect to get a five 100 times. Thus, because we observe that there is 1 chance in 6 of getting a five when a fair die is thrown, we say that the probability of this event is 1/6. But long before the relative frequentist theory came to dominate our approach to probability and statistics, a more general Bayesian approach to probability and statistics had been pioneered by Thomas Bayes (1702–1761). The classical approach is based on objective information culled from experimental observations, but Bayes allowed subjective assessments of probabilities to be made, calling these assessments the prior beliefs. In fact, the classical approach is just a simple case of Bayesian probability and statistics, where there is no subjective information and so the prior distribution is uniform.

pages: 478 words: 146,480

Pirate Cinema by Cory Doctorow


airport security, citation needed, Internet Archive, place-making, QR code, smart cities, Thomas Bayes

Brings up the grass a treat, as you can see." He gestured at the rolling lawns to one side of the ancient, mossy, fenced-in headstones. "Nonconformist cemetery," he went on, leading me deeper. "Unconsecrated ground. Lots of interesting folks buried here. You got your writers: like John Bunyon who wrote Pilgrims Progress. You got your philosophers, like Thomas Hardy. And some real maths geniuses, like old Thomas Bayes --" He pointed to a low, mossy tomb. "He invented a branch of statistics that got built into every spam filter, a couple hundred years after they buried him." He sat down on a bench. It was after mid-day now, and only a few people were eating lunch around us, none close enough to overhear us. "It's a grand life as a gentleman adventurer," he said. "Nothing to do all day but pluck choice morsels out of the bin and read the signboards the local historical society puts up in the graveyard."

pages: 396 words: 117,149

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World by Pedro Domingos


3D printing, Albert Einstein, Amazon Mechanical Turk, Arthur Eddington, basic income, Bayesian statistics, Benoit Mandelbrot, bioinformatics, Black Swan, Brownian motion, cellular automata, Claude Shannon: information theory, combinatorial explosion, computer vision, constrained optimization, correlation does not imply causation, creative destruction, crowdsourcing, Danny Hillis, data is the new oil, double helix, Douglas Hofstadter, Erik Brynjolfsson, experimental subject, Filter Bubble, future of work, global village, Google Glasses, Gödel, Escher, Bach, information retrieval, job automation, John Markoff, John Snow's cholera map, John von Neumann, Joseph Schumpeter, Kevin Kelly, lone genius, mandelbrot fractal, Mark Zuckerberg, Moneyball by Michael Lewis explains big data, Narrative Science, Nate Silver, natural language processing, Netflix Prize, Network effects, NP-complete, off grid, P = NP, PageRank, pattern recognition, phenotype, planetary scale, pre–internet, random walk, Ray Kurzweil, recommendation engine, Richard Feynman, Richard Feynman, Second Machine Age, self-driving car, Silicon Valley, speech recognition, statistical model, Stephen Hawking, Steven Levy, Steven Pinker, superintelligent machines, the scientific method, The Signal and the Noise by Nate Silver, theory of mind, Thomas Bayes, transaction costs, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, white flight, zero-sum game

Once we know how to do all these things, we’ll be ready to learn the Bayesian way. For Bayesians, learning is “just” another application of Bayes’ theorem, with whole models as the hypotheses and the data as the evidence: as you see more data, some models become more likely and some less, until ideally one model stands out as the clear winner. Bayesians have invented fiendishly clever kinds of models. So let’s get started. Thomas Bayes was an eighteenth-century English clergyman who, without realizing it, became the center of a new religion. You may well ask how that could happen, until you notice that it happened to Jesus, too: Christianity as we know it was invented by Saint Paul, while Jesus saw himself as the pinnacle of the Jewish faith. Similarly, Bayesianism as we know it was invented by Pierre-Simon de Laplace, a Frenchman who was born five decades after Bayes.

pages: 309 words: 114,984

The Digital Doctor: Hope, Hype, and Harm at the Dawn of Medicine’s Computer Age by Robert Wachter


activist fund / activist shareholder / activist investor, Affordable Care Act / Obamacare, AI winter, Airbnb, Atul Gawande, Captain Sullenberger Hudson, Checklist Manifesto, Chuck Templeton: OpenTable, Clayton Christensen, collapse of Lehman Brothers, computer age, creative destruction, crowdsourcing, deskilling,, Erik Brynjolfsson, everywhere but in the productivity statistics, Firefox, Frank Levy and Richard Murnane: The New Division of Labor, Google Glasses, Ignaz Semmelweis: hand washing, Internet of things, job satisfaction, Joseph Schumpeter, knowledge worker, lifelogging, medical malpractice, medical residency, Menlo Park, minimum viable product, natural language processing, Network effects, Nicholas Carr, obamacare, pattern recognition, peer-to-peer, personalized medicine,, Productivity paradox, Ralph Nader, RAND corporation, Second Machine Age, self-driving car, Silicon Valley, Silicon Valley startup, six sigma, Skype, Snapchat, software as a service, Steve Jobs, Steven Levy, the payments system, The Wisdom of Crowds, Thomas Bayes, Toyota Production System, Uber for X, US Airways Flight 1549, Watson beat the top human players on Jeopardy!, Yogi Berra

This is the part of diagnostic reasoning that beginners find most vexing, since they lack the foundational knowledge to understand why their teacher focused so intently on one nugget of information and all but ignored others that, to the novice, seemed equally crucial. How do the great diagnosticians make such choices? We now recognize this as a relatively intuitive version of Bayes’ theorem. Developed by the eighteenth-century British theologian-turned-mathematician Thomas Bayes, this theorem (often ignored by students because it is taught to them with the dryness of a Passover matzo) is the linchpin of clinical reasoning. In essence, Bayes’ theorem says that any medical test must be interpreted from two perspectives. The first: How accurate is the test—that is, how often does it give right or wrong answers? The second: How likely is it that this patient has the disease the test is looking for?

pages: 537 words: 144,318

The Invisible Hands: Top Hedge Fund Traders on Bubbles, Crashes, and Real Money by Steven Drobny


Albert Einstein, Asian financial crisis, asset allocation, asset-backed security, backtesting, banking crisis, Bernie Madoff, Black Swan, Bretton Woods, BRICs, British Empire, business process, capital asset pricing model, capital controls, central bank independence, collateralized debt obligation, commoditize, Commodity Super-Cycle, commodity trading advisor, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, currency peg, debt deflation, diversification, diversified portfolio, equity premium, family office, fiat currency, fixed income, follow your passion, full employment, George Santayana, Hyman Minsky, implied volatility, index fund, inflation targeting, interest rate swap, inventory management, invisible hand, London Interbank Offered Rate, Long Term Capital Management, market bubble, market fundamentalism, market microstructure, moral hazard, Myron Scholes, North Sea oil, open economy, peak oil, pension reform, Ponzi scheme, prediction markets, price discovery process, price stability, private sector deleveraging, profit motive, purchasing power parity, quantitative easing, random walk, reserve currency, risk tolerance, risk-adjusted returns, risk/return, savings glut, selection bias, Sharpe ratio, short selling, sovereign wealth fund, special drawing rights, statistical arbitrage, stochastic volatility, survivorship bias, The Great Moderation, Thomas Bayes, time value of money, too big to fail, transaction costs, unbiased observer, value at risk, Vanguard fund, yield curve, zero-sum game

We shrink my expected returns towards zero rather than towards some equilibrium model forecast, the latter of which is more appropriate given our macro focus. We then input that adjusted expected return into the Titanic funnel, which assesses how much it will lose in a variety of cataclysms, giving me the recommended position. It is important to note that I am not putting trades on just to achieve the Titanic loss number. I am just looking for mispricings. Bayesian Methods The term Bayesian refers to the work Thomas Bayes, who proved a specific case of the now eponymous theorem, published after his death in 1761. The Bayesian interpretation of probability can be seen as a form of logic that allows for analysis of uncertain statements. To evaluate the probability of a hypothesis, Bayes’ theorem compares probabilities before and after the existence of new data. Unlike other methods for analyzing hypotheses, which attempt to reject or accept a statement, the Bayesian view seeks to assign dynamic probabilities that depend on the existence of relevant information.

pages: 654 words: 191,864

Thinking, Fast and Slow by Daniel Kahneman


Albert Einstein, Atul Gawande, availability heuristic, Bayesian statistics, Black Swan, Cass Sunstein, Checklist Manifesto, choice architecture, cognitive bias, complexity theory, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, delayed gratification, demand response, endowment effect, experimental economics, experimental subject, Exxon Valdez, feminist movement, framing effect, hindsight bias, index card, information asymmetry, job satisfaction, John von Neumann, Kenneth Arrow, libertarian paternalism, loss aversion, medical residency, mental accounting, meta analysis, meta-analysis, nudge unit, pattern recognition, Paul Samuelson, pre–internet, price anchoring, quantitative trading / quantitative finance, random walk, Richard Thaler, risk tolerance, Robert Metcalfe, Ronald Reagan, The Chicago School, The Wisdom of Crowds, Thomas Bayes, transaction costs, union organizing, Walter Mischel, Yom Kippur War

And if you believe that there is a 30% chance that candidate X will be elected president, and an 80% chance that he will be reelected if he wins the first time, then you must believe that the chances that he will be elected twice in a row are 24%. The relevant “rules” for cases such as the Tom W problem are provided by Bayesian statistics. This influential modern approach to statistics is named after an English minister of the eighteenth century, the Reverend Thomas Bayes, who is credited with the first major contribution to a large problem: the logic of how people should change their mind in the light of evidence. Bayes’s rule specifies how prior beliefs (in the examples of this chapter, base rates) should be combined with the diagnosticity of the evidence, the degree to which it favors the hypothesis over the alternative. For example, if you believe that 3% of graduate students are enrolled in computer science (the base rate), and you also believe that the description of Tom W is 4 times more likely for a graduate student in that field than in other fields, then Bayes’s rule says you must believe that the probability that Tom W is a computer scientist is now 11%.

pages: 651 words: 180,162

Antifragile: Things That Gain From Disorder by Nassim Nicholas Taleb


Air France Flight 447, Andrei Shleifer, banking crisis, Benoit Mandelbrot, Berlin Wall, Black Swan, Chuck Templeton: OpenTable, commoditize, creative destruction, credit crunch, Daniel Kahneman / Amos Tversky, David Ricardo: comparative advantage, discrete time, double entry bookkeeping, Emanuel Derman, epigenetics, financial independence, Flash crash, Gary Taubes, George Santayana, Gini coefficient, Henri Poincaré, high net worth, hygiene hypothesis, Ignaz Semmelweis: hand washing, informal economy, invention of the wheel, invisible hand, Isaac Newton, James Hargreaves, Jane Jacobs, joint-stock company, joint-stock limited liability company, Joseph Schumpeter, Kenneth Arrow, knowledge economy, Lao Tzu, Long Term Capital Management, loss aversion, Louis Pasteur, mandelbrot fractal, Marc Andreessen, meta analysis, meta-analysis, microbiome, money market fund, moral hazard, mouse model, Myron Scholes, Norbert Wiener, pattern recognition, Paul Samuelson, placebo effect, Ponzi scheme, principal–agent problem, purchasing power parity, quantitative trading / quantitative finance, Ralph Nader, random walk, Ray Kurzweil, rent control, Republic of Letters, Ronald Reagan, Rory Sutherland, selection bias, Silicon Valley, six sigma, spinning jenny, statistical model, Steve Jobs, Steven Pinker, Stewart Brand, stochastic process, stochastic volatility, The Great Moderation, the new new thing, The Wealth of Nations by Adam Smith, Thomas Bayes, Thomas Malthus, too big to fail, transaction costs, urban planning, Vilfredo Pareto, Yogi Berra, Zipf's Law

There were two main sources of technical knowledge and innovation in the nineteenth and early twentieth centuries: the hobbyist and the English rector, both of whom were generally in barbell situations. An extraordinary proportion of work came out of the rector, the English parish priest with no worries, erudition, a large or at least comfortable house, domestic help, a reliable supply of tea and scones with clotted cream, and an abundance of free time. And, of course, optionality. The enlightened amateur, that is. The Reverends Thomas Bayes (as in Bayesian probability) and Thomas Malthus (Malthusian overpopulation) are the most famous. But there are many more surprises, cataloged in Bill Bryson’s Home, in which the author found ten times more vicars and clergymen leaving recorded traces for posterity than scientists, physicists, economists, and even inventors. In addition to the previous two giants, I randomly list contributions by country clergymen: Rev.

pages: 685 words: 203,949

The Organized Mind: Thinking Straight in the Age of Information Overload by Daniel J. Levitin


airport security, Albert Einstein, Amazon Mechanical Turk, Anton Chekhov, Bayesian statistics, big-box store, business process, call centre, Claude Shannon: information theory, cloud computing, cognitive bias, complexity theory, computer vision, conceptual framework, correlation does not imply causation, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, delayed gratification, Donald Trump,, epigenetics, Eratosthenes, Exxon Valdez, framing effect, friendly fire, fundamental attribution error, Golden Gate Park, Google Glasses, haute cuisine, impulse control, index card, indoor plumbing, information retrieval, invention of writing, iterative process, jimmy wales, job satisfaction, Kickstarter, life extension, meta analysis, meta-analysis, more computing power than Apollo, Network effects, new economy, Nicholas Carr, optical character recognition, Pareto efficiency, pattern recognition, phenotype, placebo effect, pre–internet, profit motive, randomized controlled trial, Rubik’s Cube, Skype, Snapchat, statistical model, Steve Jobs, supply-chain management, the scientific method, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, theory of mind, Thomas Bayes, Turing test, ultimatum game, zero-sum game

Very unlikely. So this shows we’re capable of using base rate information when events are extremely unlikely. It’s when they’re only mildly unlikely that our brains freeze up. Organizing our decisions requires that we combine the base rate information with other relevant diagnostic information. This type of reasoning was discovered in the eighteenth century by the mathematician and Presbyterian minister Thomas Bayes, and bears his name: Bayes’s rule. Bayes’s rule allows us to refine estimates. For example, we read that roughly half of marriages end in divorce. But we can refine that estimate if we have additional information, such as the age, religion, or location of the people involved, because the 50% figure holds only for the aggregate of all people. Some subpopulations of people have higher divorce rates than others.

pages: 698 words: 198,203

The Stuff of Thought: Language as a Window Into Human Nature by Steven Pinker

airport security, Albert Einstein, Bob Geldof, colonial rule, conceptual framework, correlation does not imply causation, Daniel Kahneman / Amos Tversky, David Brooks, Douglas Hofstadter,, experimental subject, fudge factor, George Santayana, loss aversion, luminiferous ether, Norman Mailer, Richard Feynman, Richard Feynman, Ronald Reagan, Sapir-Whorf hypothesis, science of happiness, speech recognition, stem cell, Steven Pinker, Thomas Bayes, Thorstein Veblen, traffic fines, urban renewal, Yogi Berra

The world is a tissue of causes and effects that criss and cross in tangled patterns. The embarrassments for Hume’s two theories of causation (conjunction and counterfactuals) can be diagrammed as a family of networks in which the lines fan in or out or loop around, as in the diagram on the following page. One solution to the webbiness of causation is a technique in artificial intelligence called Causal Bayes Networks.120 (They are named for Thomas Bayes, whose eponymous theorem shows how to calculate the probability of some condition from its prior plausibility and the likelihood that it led to some observed symptoms.) A modeler chooses a set of variables (amount of coffee drunk, amount of exercise, presence of heart disease, and so on), draws arrows between causes and their effects, and labels each arrow with a number representing the strength of the causal influence (the increase or decrease in the likelihood of the effect, given the presence of the cause).

Data Mining: Concepts and Techniques: Concepts and Techniques by Jiawei Han, Micheline Kamber, Jian Pei


bioinformatics, business intelligence, business process, Claude Shannon: information theory, cloud computing, computer vision, correlation coefficient, cyber-physical system, database schema, discrete time, distributed generation, finite state, information retrieval, iterative process, knowledge worker, linked data, natural language processing, Netflix Prize, Occam's razor, pattern recognition, performance metric, phenotype, random walk, recommendation engine, RFID, semantic web, sentiment analysis, speech recognition, statistical model, stochastic process, supply-chain management, text mining, thinkpad, Thomas Bayes, web application

Naïve Bayesian classifiers assume that the effect of an attribute value on a given class is independent of the values of the other attributes. This assumption is called class-conditional independence. It is made to simplify the computations involved and, in this sense, is considered “naïve." Section 8.3.1 reviews basic probability notation and Bayes’ theorem. In Section 8.3.2 you will learn how to do naïve Bayesian classification. 8.3.1. Bayes’ Theorem Bayes’ theorem is named after Thomas Bayes, a nonconformist English clergyman who did early work in probability and decision theory during the 18th century. Let X be a data tuple. In Bayesian terms, X is considered “evidence.” As usual, it is described by measurements made on a set of n attributes. Let H be some hypothesis such as that the data tuple X belongs to a specified class C. For classification problems, we want to determine , the probability that the hypothesis H holds given the “evidence” or observed data tuple X.