Reflections on Trusting Trust

6 results back to index

Your Computer Is on Fire by Thomas S. Mullaney, Benjamin Peters, Mar Hicks, Kavita Philip

"Susan Fowler" uber, 2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, A Declaration of the Independence of Cyberspace, affirmative action, Airbnb, algorithmic bias, AlphaGo, AltaVista, Amazon Mechanical Turk, Amazon Web Services, American Society of Civil Engineers: Report Card, An Inconvenient Truth, Asilomar, autonomous vehicles, Big Tech, bitcoin, Bletchley Park, blockchain, Boeing 737 MAX, book value, British Empire, business cycle, business process, Californian Ideology, call centre, Cambridge Analytica, carbon footprint, Charles Babbage, cloud computing, collective bargaining, computer age, computer vision, connected car, corporate governance, corporate social responsibility, COVID-19, creative destruction, cryptocurrency, dark matter, data science, Dennis Ritchie, deskilling, digital divide, digital map, don't be evil, Donald Davies, Donald Trump, Edward Snowden, en.wikipedia.org, European colonialism, fake news, financial innovation, Ford Model T, fulfillment center, game design, gentrification, George Floyd, glass ceiling, global pandemic, global supply chain, Grace Hopper, hiring and firing, IBM and the Holocaust, industrial robot, informal economy, Internet Archive, Internet of things, Jeff Bezos, job automation, John Perry Barlow, Julian Assange, Ken Thompson, Kevin Kelly, Kickstarter, knowledge economy, Landlord’s Game, Lewis Mumford, low-wage service sector, M-Pesa, Mark Zuckerberg, mass incarceration, Menlo Park, meta-analysis, mobile money, moral panic, move fast and break things, Multics, mutually assured destruction, natural language processing, Neal Stephenson, new economy, Norbert Wiener, off-the-grid, old-boy network, On the Economy of Machinery and Manufactures, One Laptop per Child (OLPC), packet switching, pattern recognition, Paul Graham, pink-collar, pneumatic tube, postindustrial economy, profit motive, public intellectual, QWERTY keyboard, Ray Kurzweil, Reflections on Trusting Trust, Report Card for America’s Infrastructure, Salesforce, sentiment analysis, Sheryl Sandberg, Silicon Valley, Silicon Valley ideology, smart cities, Snapchat, speech recognition, SQL injection, statistical model, Steve Jobs, Stewart Brand, tacit knowledge, tech worker, techlash, technoutopianism, telepresence, the built environment, the map is not the territory, Thomas L Friedman, TikTok, Triangle Shirtwaist Factory, undersea cable, union organizing, vertical integration, warehouse robotics, WikiLeaks, wikimedia commons, women in the workforce, Y2K

In his acceptance speech for this award, titled “Reflections on Trusting Trust,”2 Thompson reminds the audience that he hadn’t worked actively on Unix in many years. After doing this act of performative humility, Thompson devotes his speech to a description of “the cutest program he ever wrote.” This program was an implementation of a method of creating Trojan horse programs—software with secret backdoors, which the nefarious creator of the Trojan horse could then use to hijack computer systems on which the software was installed. I will refer to the method that Thompson outlines in “Reflections on Trusting Trust” as the “Thompson hack” for the remainder of this chapter.3 Nevertheless, it is useful to note that what I call the Thompson hack was not wholly the result of Thompson’s original research.

…

However, it also holds its own inherent interest, since this layer of translation standing between human-friendlier programming languages and code that computers can run is one of the deepest elements in the deep-layered software stack that allows for the creation of software platforms. The primer on programming language hierarchies is followed by a close read of “Reflections on Trusting Trust.” Although this material is fairly technical, enough apparatus is provided for nonprogramming readers to understand both the key turning points of Thompson’s explanation of the hack and the significance of the hack in understanding software platforms. Once the technical work of explicating the Thompson hack’s methodology is finished, I turn to analyzing the social implications of the hack, and how those implications may differ from the ones that Thompson himself proposes in “Reflections.”

…

This game involved writing programs that would, when compiled and run, produce complete listings of their own source code as output. The winner would be the programmer who had produced the shortest self-replicating program. The method Thompson used to produce a self-replicating pronoun that he presents in “Reflections on Trusting Trust” is inspired by analytic philosopher W. V. O. Quine’s variant of the liar’s paradox. Most versions of this paradox, like the well-known “this sentence is false” formulation, contain demonstrative words referring to the sentences themselves. In “this sentence is false,” the demonstrative word is “this.”

Turing's Vision: The Birth of Computer Science by Chris Bernhardt

Ada Lovelace, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, Andrew Wiles, Bletchley Park, British Empire, cellular automata, Charles Babbage, Claude Shannon: information theory, complexity theory, Computing Machinery and Intelligence, Conway's Game of Life, discrete time, Douglas Hofstadter, Georg Cantor, Gödel, Escher, Bach, Henri Poincaré, Internet Archive, Jacquard loom, John Conway, John von Neumann, Joseph-Marie Jacquard, Ken Thompson, Norbert Wiener, Paul Erdős, Reflections on Trusting Trust, Turing complete, Turing machine, Turing test, Von Neumann architecture

“Symbolic Analysis of Relay and Switching Circuits,” Transactions American Institute of Electrical Engineers, vol. 57, 1938, pp. 38–80. [45] Sipser, Michael. Introduction to the Theory of Computation, Cengage Learning, 2012. [46] Soare, Robert. “Formalism and intuition in computability,” Phil. Trans. R, soc. A, (2012) 370, pp. 3277–3304. [47] Thompson, Ken. “Reflections on Trusting Trust,” Communications of the ACM, August 1984, vol. 27, no. 8, pp. 761–763. [48] Tibor, Radó. “On non-computable functions,” Bell System Technical Journal 41 (3) pp. 877–884, 1962. [49] Turing, Alan. “Computing machinery and intelligence,” Mind 1950, 59, 433–460. [50] Turing, Alan. “On Computable Numbers, with an Application to the Entscheidungsproblem,” Proceedings of the London Mathematical Society, Series 2, 42 (1936–7), pp. 230–265

…

See also Acceptance problem; Blank tape problem; Halting problem Universal computer/machine, 12, 87, 91 u-substitution, 62 von Neumann, John, 26, 97, 148, 155, 164 von Neumann architecture, 97, 155 Whitehead, Alfred North, Principia Mathematica, 7, 8, 10, 16 Wiener, Norbert, 26 Williams, Frederick, 156 Wolfram, Stephen, 85, 103, 164 Zuse, Konrad, 154 1 “Reflections on Trusting Trust” was presented by Ken Thompson in 1983. It was published in the Communications of the ACM and is widely available on the web.

This Is How They Tell Me the World Ends: The Cyberweapons Arms Race by Nicole Perlroth

4chan, active measures, activist lawyer, air gap, Airbnb, Albert Einstein, Apollo 11, barriers to entry, Benchmark Capital, Bernie Sanders, Big Tech, bitcoin, Black Lives Matter, blood diamond, Boeing 737 MAX, Brexit referendum, Brian Krebs, Citizen Lab, cloud computing, commoditize, company town, coronavirus, COVID-19, crony capitalism, crowdsourcing, cryptocurrency, dark matter, David Vincenzetti, defense in depth, digital rights, disinformation, don't be evil, Donald Trump, driverless car, drone strike, dual-use technology, Edward Snowden, end-to-end encryption, failed state, fake news, false flag, Ferguson, Missouri, Firefox, gender pay gap, George Floyd, global pandemic, global supply chain, Hacker News, index card, information security, Internet of things, invisible hand, Jacob Appelbaum, Jeff Bezos, John Markoff, Ken Thompson, Kevin Roose, Laura Poitras, lockdown, Marc Andreessen, Mark Zuckerberg, mass immigration, Menlo Park, MITM: man-in-the-middle, moral hazard, Morris worm, move fast and break things, mutually assured destruction, natural language processing, NSO Group, off-the-grid, offshore financial centre, open borders, operational security, Parler "social media", pirate software, purchasing power parity, race to the bottom, RAND corporation, ransomware, Reflections on Trusting Trust, rolodex, Rubik’s Cube, Russian election interference, Sand Hill Road, Seymour Hersh, Sheryl Sandberg, side project, Silicon Valley, Skype, smart cities, smart grid, South China Sea, Steve Ballmer, Steve Bannon, Steve Jobs, Steven Levy, Stuxnet, supply-chain attack, TED Talk, the long tail, the scientific method, TikTok, Tim Cook: Apple, undersea cable, unit 8200, uranium enrichment, web application, WikiLeaks, zero day, Zimmermann PGP

In dissecting those chips down to the very bit, Gosler could see that these advancements—and the complexity they introduced—would only create more room for error, malfunction, and eventually enemy subversion and attack. The previous year, Gosler had heard a famous lecture by Ken Thompson. Thompson, who had won the 1983 Turing Award for cocreating the Unix operating system, used his turn at the lectern to share his concerns on where technology was headed. He’d titled his lecture “Reflections on Trusting Trust,” and his conclusion was this: unless you wrote the source code yourself, you could never be confident that a computer program wasn’t a Trojan horse. Thompson had perfectly articulated what Gosler knew to be true. But by the time Gosler listened to Thompson’s lecture, he could see that the predicament was getting exponentially worse.

…

The reference to Sandia’s role in developing 97 percent of America’s non-nuclear weapons components is available on Sandia’s website: “Evaluating Nuclear Weapons: A Key Sandia Mission.” Eric Schlosser provided an entertaining, and disturbing, account of America’s nuclear weapons accidents in his 2013 book, Command and Control: Nuclear Weapons, the Damascus Accident and the Illusion of Safety (Penguin Press). Ken Thompson’s 1984 Turing Award speech, “Reflections on Trusting Trust,” is available here: www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ReflectionsonTrustingTrust.pdf. Gosler’s Chaperon Experiments were also detailed in a 2016 dissertation by Craig J. Weiner, at George Mason University, titled: “Penetrate, Exploit, Disrupt, Destroy: The Rise of Computer Network Operations as a Major Military Innovation.”

…

., warnings to the, here Pwn2Own hacking contest, here Qatar, here Q Group (NSA), here Qualcomm, here ransomware attacks, here, here, here Ratcliffe, John, here Rather, Dan, here Raymond, Eric S., here Raytheon, here Reagan, Ronald, here, here, here, here Reckitt Benckiser, here Reddit, here “Reflections on Trusting Trust” (Thompson), here Retz, Dave, here Rhodes, Benjamin, here Rice, Alex, here, here, here Richarte, Gerardo (Gera), here, here Rizzo, Juliano, here, here Robertson, Pat, here Rogen, Seth, here, here Rogers, Michael, here Romney, Mitt, here room taps, here Rosenberg, Paul, here Rosneft (Russia), here, here RSA, here, here, here Russia break-up of the Soviet Union, here cyberespionage, here disinformation campaigns, here economy, here election interference (2016), here, here, here, here, here, here, here, here election interference (2020), here EternalBlue, use of, here kompromat, here, here ransomware attacks, here Ukraine, invasion of, here U.S. cyberattacks on, here U.S. grid, vulnerability to, here, here U.S. grid attacks in, here U.S. sanctions, here voter registration system hacks, here, here, here WannaCry ransomware in, here Russia, cyberattacks DNC, here, here, here, here, here, here, here, here outsourcing, here range of, here State Department, here, here TrickBot, here, here, here, here, here warning issued by, here White House, here Russia, cyberattacks in Ukraine election interference, here, here elections, here election systems, here factors limiting, here individuals, here infrastructure, here, here, here, here, here, here, here media attacks, here nuclear plants, here purpose, here al-Saadan, Abdullah, here Sabien, Jimmy, here, here, here Safari, here, here, here Safari (Apple), here Sagan, Carl, here Sahin, Tamer, here Said, Missoum, here Salehi, Ali Akbar, here Samba, here Sandberg, Sheryl, here Sanders, Bernie, here, here, here, here Sandia National Labs, here, here, here Sandler, Adam, here Sands casinos, here, here Sandworm (Russia), here, here Sanger, David, here, here, here, here, here, here, here, here, here, here, here, here Saudi Arabia, here, here, here, here, here Schlesinger, James, here Schmidt, Eric, here, here, here, here, here Schmidt, Howard, here Schneier, Bruce, here Schroeder, Gerhard, here Schulte, John, here SecurityFocus, here September here, 2001 terrorist attacks, here, here, here, here, here Sequoia Capital, here Shadow Brokers, here, here, here, here, here, here, here, here Shane, Scott, here, here, here, here, here, here, here Shwedo, Bradford “B.

Fancy Bear Goes Phishing: The Dark History of the Information Age, in Five Extraordinary Hacks by Scott J. Shapiro

3D printing, 4chan, active measures, address space layout randomization, air gap, Airbnb, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, availability heuristic, Bernie Sanders, bitcoin, blockchain, borderless world, Brian Krebs, business logic, call centre, carbon tax, Cass Sunstein, cellular automata, cloud computing, cognitive dissonance, commoditize, Compatible Time-Sharing System, Computing Machinery and Intelligence, coronavirus, COVID-19, CRISPR, cryptocurrency, cyber-physical system, Daniel Kahneman / Amos Tversky, Debian, Dennis Ritchie, disinformation, Donald Trump, double helix, Dr. Strangelove, dumpster diving, Edward Snowden, en.wikipedia.org, Evgeny Morozov, evil maid attack, facts on the ground, false flag, feminist movement, Gabriella Coleman, gig economy, Hacker News, independent contractor, information security, Internet Archive, Internet of things, invisible hand, John Markoff, John von Neumann, Julian Assange, Ken Thompson, Larry Ellison, Laura Poitras, Linda problem, loss aversion, macro virus, Marc Andreessen, Mark Zuckerberg, Menlo Park, meta-analysis, Minecraft, Morris worm, Multics, PalmPilot, Paul Graham, pirate software, pre–internet, QWERTY keyboard, Ralph Nader, RAND corporation, ransomware, Reflections on Trusting Trust, Richard Stallman, Richard Thaler, Ronald Reagan, Satoshi Nakamoto, security theater, Shoshana Zuboff, side hustle, Silicon Valley, Skype, SoftBank, SQL injection, Steve Ballmer, Steve Jobs, Steven Levy, Stuxnet, supply-chain attack, surveillance capitalism, systems thinking, TaskRabbit, tech billionaire, tech worker, technological solutionism, the Cathedral and the Bazaar, the new new thing, the payments system, Turing machine, Turing test, Unsafe at Any Speed, vertical integration, Von Neumann architecture, Wargames Reagan, WarGames: Global Thermonuclear War, Wayback Machine, web application, WikiLeaks, winner-take-all economy, young professional, zero day, éminence grise

This statute limited the criminal offense to three specific scenarios—unauthorized access to obtain national security secrets, personal financial records from financial institutions or credit agencies, and hacking into government computers. devoted his lecture to cybersecurity: Kenneth Thompson, “Reflections on Trusting Trust,” Communications of the ACM, August 1984, https://www.cs.cmu.edu/~rdriley/487/papers/Thompson_1984_ ReflectionsonTrustingTrust.pdf. The Turing lecture series was inaugurated in 1967. air force testers: Karger and Schell provided the first public description of the problem that compilers can insert malicious code into themselves.

…

do the same to UNIX: David Wheeler proposed a countermeasure against the Thompson attack using two different compilers, in David Wheeler, Fully Countering Trusting Trust Through Diverse Double-Compiling (PhD diss., George Mason University, 2009), https://dwheeler.com/trusting-trust/dissertation/html/wheeler-trusting-trust-ddc.html. “only program you can truly trust”: Thompson, “Reflections on Trusting Trust.” appearing on: Patrick was also a witness at the congressional cybersecurity hearings. When asked by a member of the subcommittee whether WarGames was an inspiration, Patrick disappointed: “That didn’t instigate us at all.” Many hackers, however, have since claimed that the movie was indeed their first inspiration.

Protocol: how control exists after decentralization by Alexander R. Galloway

Ada Lovelace, airport security, Alvin Toffler, Berlin Wall, bioinformatics, Bretton Woods, Charles Babbage, computer age, Computer Lib, Craig Reynolds: boids flock, Dennis Ritchie, digital nomad, discovery of DNA, disinformation, Donald Davies, double helix, Douglas Engelbart, Douglas Engelbart, easy for humans, difficult for computers, Fall of the Berlin Wall, Free Software Foundation, Grace Hopper, Hacker Ethic, Hans Moravec, informal economy, John Conway, John Markoff, John Perry Barlow, Ken Thompson, Kevin Kelly, Kickstarter, late capitalism, Lewis Mumford, linear programming, macro virus, Marshall McLuhan, means of production, Menlo Park, moral panic, mutually assured destruction, Norbert Wiener, old-boy network, OSI model, packet switching, Panopticon Jeremy Bentham, phenotype, post-industrial society, profit motive, QWERTY keyboard, RAND corporation, Ray Kurzweil, Reflections on Trusting Trust, RFC: Request For Comment, Richard Stallman, semantic web, SETI@home, stem cell, Steve Crocker, Steven Levy, Stewart Brand, Ted Nelson, telerobotics, The future is already here, the market place, theory of mind, urban planning, Vannevar Bush, Whole Earth Review, working poor, Yochai Benkler

Douglas McIlroy, head of the Computing Techniques Research Department at Bell Labs, and a program called Worm created by John Shoch (and Jon Hupp) of Xerox Palo Alto Research Center. See A. K. Dewdney, “Computer Recreations,” Scientific American, March 1984, p. 22. For more on Shoch and Hupp, see “The Worm Programs,” Communications of the ACM, March 1982. Many attribute the worm concept to the science fiction novel Shockwave Rider by John Brunner. 20. Ken Thompson, “Reflections on Trusting Trust,” in Computers Under Attack: Intruders, Worms, and Viruses, ed. Peter Denning (New York: ACM, 1990), p. 98. 21. Dewdney, “Computer Recreations,” p. 14. 22. Jon A. Rochlis and Mark W. Eichin, “With Microscope and Tweezers: The Worm from MIT’s Perspective,” in Computers Under Attack: Intruders, Worms, and Viruses, ed.

Dark Mirror: Edward Snowden and the Surveillance State by Barton Gellman

4chan, A Declaration of the Independence of Cyberspace, Aaron Swartz, active measures, air gap, Anton Chekhov, Big Tech, bitcoin, Cass Sunstein, Citizen Lab, cloud computing, corporate governance, crowdsourcing, data acquisition, data science, Debian, desegregation, Donald Trump, Edward Snowden, end-to-end encryption, evil maid attack, financial independence, Firefox, GnuPG, Google Hangouts, housing justice, informal economy, information security, Jacob Appelbaum, job automation, John Perry Barlow, Julian Assange, Ken Thompson, Laura Poitras, MITM: man-in-the-middle, national security letter, off-the-grid, operational security, planetary scale, private military company, ransomware, Reflections on Trusting Trust, Robert Gordon, Robert Hanssen: Double agent, rolodex, Ronald Reagan, Saturday Night Live, seminal paper, Seymour Hersh, Silicon Valley, Skype, social graph, standardized shipping container, Steven Levy, TED Talk, telepresence, the long tail, undersea cable, Wayback Machine, web of trust, WikiLeaks, zero day, Zimmermann PGP

See also Lawrence Joffe, “Abu Musab al-Zarqawi Obituary,” Guardian, June 8, 2006, at https://perma.cc/8T2C-NZFP. fist-bumping status report: On file with author. taking part in a criminal conspiracy: See chapter 7. Eventually he agreed to breakfast: James R. Clapper, interview with author, August 17, 2018. as long ago as 1984: Kenneth Thompson, “Reflections on Trusting Trust,” Turing Award lecture, reproduced in Communications of the ACM, August 1984, at https://perma.cc/NL2L-7JX3. the Gemalto gambit: This story came to light in Jeremy Scahill and Josh Begley, “The Great SIM Heist,” Intercept, February 19, 2015, https://theintercept.com/2015/02/19/great-sim-heist/.

Dawn of the Code War: America's Battle Against Russia, China, and the Rising Global Cyber Threat by John P. Carlin, Garrett M. Graff

1960s counterculture, A Declaration of the Independence of Cyberspace, Aaron Swartz, air gap, Andy Carvin, Apple II, Bay Area Rapid Transit, bitcoin, Brian Krebs, business climate, cloud computing, cotton gin, cryptocurrency, data acquisition, Deng Xiaoping, disinformation, driverless car, drone strike, dual-use technology, eat what you kill, Edward Snowden, fake news, false flag, Francis Fukuyama: the end of history, Hacker Ethic, information security, Internet of things, James Dyson, Jeff Bezos, John Gilmore, John Markoff, John Perry Barlow, Ken Thompson, Kevin Roose, Laura Poitras, Mark Zuckerberg, Menlo Park, millennium bug, Minecraft, Mitch Kapor, moral hazard, Morris worm, multilevel marketing, Network effects, new economy, Oklahoma City bombing, out of africa, packet switching, peer-to-peer, peer-to-peer model, performance metric, RAND corporation, ransomware, Reflections on Trusting Trust, Richard Stallman, Robert Metcalfe, Ronald Reagan, Saturday Night Live, self-driving car, shareholder value, side project, Silicon Valley, Silicon Valley startup, Skype, Snapchat, South China Sea, Steve Crocker, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, Stuxnet, The Hackers Conference, Tim Cook: Apple, trickle-down economics, Wargames Reagan, Whole Earth Catalog, Whole Earth Review, WikiLeaks, Y2K, zero day, zero-sum game

Nikki Finke, “A University Professor’s ‘Startling’ Experiments Began It All,” Los Angeles Times, January 31, 1988, articles.latimes.com/1988-01-31/news/vw-39340_1 _computer-virus; and “When Did the Term ‘Computer Virus’ Arise?” Scientific American, September 2, 1997, scientificamerican.com/article/when-did-the-term-compute/. 43. Michelle Slatalla and Joshua Quittner, Masters of Deception: The Gang That Ruled Cyberspace (HarperCollins, 1995), 16. 44. Ken Thompson, “Reflections on Trusting Trust,” Turing Award Lecture, Communications of the ACM, vol. 1, no. 8, 1984, www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf. 45. Linda Greenhouse, “House Approves Measure to Make Computer Fraud a Federal Crime,” New York Times, June 4, 1986, www.nytimes.com/1986/06/04/us/house-approves-measure-to-make-computer-fraud-a-federal-crime.html; Josephine Wolff, “The Hacking Law That Can’t Hack It,” Slate, September 27, 2016, www.slate.com/articles/technology/future_tense/2016/09/the_computer_fraud_and _abuse_act_turns_30_years_old.html; and Scott Mace, “Computer Bills in Works,” InfoWorld, October 14, 1985, books.google.com/books?

New York 2140 by Kim Stanley Robinson

Anthropocene, availability heuristic, back-to-the-land, Black-Scholes formula, Burning Man, central bank independence, creative destruction, credit crunch, crowdsourcing, decarbonisation, East Village, full employment, gentrification, happiness index / gross national happiness, hive mind, income inequality, invisible hand, Jane Jacobs, Ken Thompson, Kim Stanley Robinson, liquidity trap, Mason jar, mass immigration, megastructure, microbiome, music of the spheres, New Urbanism, offshore financial centre, Planet Labs, plutocrats, Ponzi scheme, precariat, quantitative easing, Reflections on Trusting Trust, rent-seeking, Social Justice Warrior, the built environment, too big to fail

A low discount rate makes the future more important, a high discount rate is dismissive of the future. —Frank Ackerman, Can We Afford the Future? The moral is obvious. You can’t trust code that you did not totally create yourself. Misguided use of a computer is no more amazing than drunk driving of an automobile. —Ken Thompson, “Reflections on Trusting Trust” A bird in the hand is worth what it will bring. noted Ambrose Bierce c) Franklin Numbers often fill my head. While waiting for my building’s morose super to free my Jesus bug from the boathouse rafters where it had spent the night, I was looking at the little waves lapping in the big doors and wondering if the Black-Scholes formula could frame their volatility.