deep learning

227 results back to index

Architects of Intelligence by Martin Ford

3D printing, agricultural Revolution, AI winter, algorithmic bias, Alignment Problem, AlphaGo, Apple II, artificial general intelligence, Asilomar, augmented reality, autonomous vehicles, backpropagation, barriers to entry, basic income, Baxter: Rethink Robotics, Bayesian statistics, Big Tech, bitcoin, Boeing 747, Boston Dynamics, business intelligence, business process, call centre, Cambridge Analytica, cloud computing, cognitive bias, Colonization of Mars, computer vision, Computing Machinery and Intelligence, correlation does not imply causation, CRISPR, crowdsourcing, DARPA: Urban Challenge, data science, deep learning, DeepMind, Demis Hassabis, deskilling, disruptive innovation, Donald Trump, Douglas Hofstadter, driverless car, Elon Musk, Erik Brynjolfsson, Ernest Rutherford, fake news, Fellow of the Royal Society, Flash crash, future of work, general purpose technology, Geoffrey Hinton, gig economy, Google X / Alphabet X, Gödel, Escher, Bach, Hans Moravec, Hans Rosling, hype cycle, ImageNet competition, income inequality, industrial research laboratory, industrial robot, information retrieval, job automation, John von Neumann, Large Hadron Collider, Law of Accelerating Returns, life extension, Loebner Prize, machine translation, Mark Zuckerberg, Mars Rover, means of production, Mitch Kapor, Mustafa Suleyman, natural language processing, new economy, Nick Bostrom, OpenAI, opioid epidemic / opioid crisis, optical character recognition, paperclip maximiser, pattern recognition, phenotype, Productivity paradox, radical life extension, Ray Kurzweil, recommendation engine, Robert Gordon, Rodney Brooks, Sam Altman, self-driving car, seminal paper, sensor fusion, sentiment analysis, Silicon Valley, smart cities, social intelligence, sparse data, speech recognition, statistical model, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Steve Wozniak, Steven Pinker, strong AI, superintelligent machines, synthetic biology, systems thinking, Ted Kaczynski, TED Talk, The Rise and Fall of American Growth, theory of mind, Thomas Bayes, Travis Kalanick, Turing test, universal basic income, Wall-E, Watson beat the top human players on Jeopardy!, women in the workforce, working-age population, workplace surveillance , zero-sum game, Zipcar

MARTIN FORD: How does all of that thinking relate to the current overwhelming focus on deep learning? Clearly, deep neural networks have transformed AI, but lately I’ve been hearing more pushback against deep learning hype, and even some suggestions that we could be facing a new AI Winter. Is deep learning really the primary path forward, or is it just one tool in the toolbox? JOSH TENENBAUM: What most people think of as deep learning is one tool in the toolbox, and a lot of deep learning people realize that too. The term “deep learning” has expanded beyond its original definition. MARTIN FORD: I would define deep learning broadly as any approach using sophisticated neural networks with lots of layers, rather than using a very technical definition involving specific algorithms like backpropagation or gradient descent.

…

Going one step further, deep learning is where we have neural networks that have many layers. There is no required minimum for a neural network to be deep, but we would usually say that two or three layers is not a deep learning network, while four or more layers is deep learning. Some deep learning networks get up to one thousand layers or more. By having many layers in deep learning, we can represent a very complex transformation between the input and output, by a composition of much simpler transformations, each represented by one of those layers in the network. The deep learning hypothesis suggests that many layers make it easier for the learning algorithm to find a predictor, to set all the connection strengths in the network so that it does a good job.

…

MARTIN FORD: Do you think there will be a backlash against all the hype surrounding deep learning when its limitations are more widely recognized? BARBARA GROSZ: I have survived numerous AI Winters in the past and I’ve come away from them feeling both fearful and hopeful. I’m fearful that people, once they see the limitations of deep learning will say, “Oh, it doesn’t really work.” But I’m hopeful that, because deep learning is so powerful for so many things, and in so many areas, that there won’t be an AI Winter around deep learning. I do think, however, that to avoid an AI Winter for deep learning, people in the field need to put deep learning in its correct place, and be clear about its limitations.

Genius Makers: The Mavericks Who Brought A. I. To Google, Facebook, and the World by Cade Metz

AI winter, air gap, Airbnb, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, AlphaGo, Amazon Robotics, artificial general intelligence, Asilomar, autonomous vehicles, backpropagation, Big Tech, British Empire, Cambridge Analytica, carbon-based life, cloud computing, company town, computer age, computer vision, deep learning, deepfake, DeepMind, Demis Hassabis, digital map, Donald Trump, driverless car, drone strike, Elon Musk, fake news, Fellow of the Royal Society, Frank Gehry, game design, Geoffrey Hinton, Google Earth, Google X / Alphabet X, Googley, Internet Archive, Isaac Newton, Jeff Hawkins, Jeffrey Epstein, job automation, John Markoff, life extension, machine translation, Mark Zuckerberg, means of production, Menlo Park, move 37, move fast and break things, Mustafa Suleyman, new economy, Nick Bostrom, nuclear winter, OpenAI, PageRank, PalmPilot, pattern recognition, Paul Graham, paypal mafia, Peter Thiel, profit motive, Richard Feynman, ride hailing / ride sharing, Ronald Reagan, Rubik’s Cube, Sam Altman, Sand Hill Road, self-driving car, side project, Silicon Valley, Silicon Valley billionaire, Silicon Valley startup, Skype, speech recognition, statistical model, stem cell, Stephen Hawking, Steve Ballmer, Steven Levy, Steven Pinker, tech worker, telemarketer, The Future of Employment, Turing test, warehouse automation, warehouse robotics, Y Combinator

he published what he called a trilogy of papers critiquing: Gary Marcus, “Deep Learning: A Critical Appraisal,” 2018, https://arxiv.org/abs/1801.00631; Gary Marcus, “In Defense of Skepticism About Deep Learning,” 2018, https://medium.com/@GaryMarcus/in-defense-of-skepticism-about-deep-learning-6e8bfd5ae0f1; Gary Marcus, “Innateness, AlphaZero, and Artificial Intelligence,” 2018, https://arxiv.org/abs/1801.05667. would eventually lead to a book: Gary Marcus and Ernest Davis, Rebooting AI: Building Artificial Intelligence We Can Trust (New York: Pantheon, 2019). he agreed that deep learning alone could not achieve true intelligence: “Artificial Intelligence Debate—Yann LeCun vs.

…

* * * — IN the spring of 2012, Geoff Hinton phoned Jitendra Malik, the University of California–Berkeley professor who had publicly attacked Andrew Ng over his claims that deep learning was the future of computer vision. Despite the success of deep learning with speech recognition, Malik and his colleagues questioned whether the technology would ever master the art of identifying images. And because he was someone who generally assumed that incoming calls were arriving from telemarketers trying to sell him something, it was surprising that he even picked up the phone. When he did, Hinton said: “I hear you don’t like deep learning.” Malik said this was true, and when Hinton asked why, Malik said there was no scientific evidence to back any claim that deep learning could outperform any other technology on computer vision.

…

Malik said this was true, and when Hinton asked why, Malik said there was no scientific evidence to back any claim that deep learning could outperform any other technology on computer vision. Hinton pointed to recent papers showing that deep learning worked well when identifying objects on multiple benchmark tests. Malik said these datasets were too old. No one cared about them. “This is not going to convince anyone who doesn’t share your ideological predilections,” he said. So Hinton asked what would convince him. At first, Malik said deep learning would have to master a European dataset called PASCAL. “PASCAL is too small,” Hinton told him. “To make this work, we need a lot of training data.

Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again by Eric Topol

"World Economic Forum" Davos, 23andMe, Affordable Care Act / Obamacare, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, algorithmic bias, AlphaGo, Apollo 11, artificial general intelligence, augmented reality, autism spectrum disorder, autonomous vehicles, backpropagation, Big Tech, bioinformatics, blockchain, Cambridge Analytica, cloud computing, cognitive bias, Colonization of Mars, computer age, computer vision, Computing Machinery and Intelligence, conceptual framework, creative destruction, CRISPR, crowdsourcing, Daniel Kahneman / Amos Tversky, dark matter, data science, David Brooks, deep learning, DeepMind, Demis Hassabis, digital twin, driverless car, Elon Musk, en.wikipedia.org, epigenetics, Erik Brynjolfsson, fake news, fault tolerance, gamification, general purpose technology, Geoffrey Hinton, George Santayana, Google Glasses, ImageNet competition, Jeff Bezos, job automation, job satisfaction, Joi Ito, machine translation, Mark Zuckerberg, medical residency, meta-analysis, microbiome, move 37, natural language processing, new economy, Nicholas Carr, Nick Bostrom, nudge unit, OpenAI, opioid epidemic / opioid crisis, pattern recognition, performance metric, personalized medicine, phenotype, placebo effect, post-truth, randomized controlled trial, recommendation engine, Rubik’s Cube, Sam Altman, self-driving car, Silicon Valley, Skinner box, speech recognition, Stephen Hawking, techlash, TED Talk, text mining, the scientific method, Tim Cook: Apple, traumatic brain injury, trolley problem, War on Poverty, Watson beat the top human players on Jeopardy!, working-age population

Bar, “Artificial Intelligence: Driving the Next Technology Cycle,” in Next Generation (Zurich: Julius Baer Group, 2017); Chollet, F., Deep Learning with Python (Shelter Island, New York: Manning, 2017); T. L. Fonseca, “What’s Happening Inside the Convolutional Neural Network? The Answer Is Convolution,” buZZrobot (2017); A. Geitgey, “Machine Learning Is Fun! Part 3: Deep Learning and Convolutional Neural Networks,” Medium (2016); Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” Nature (2015): 521(7553), 436–444; R. Raicea, “Want to Know How Deep Learning Works? Here’s a Quick Guide for Everyone,” Medium (2017); P. Voosen, “The AI Detectives,” Science (2017): 357(6346), 22–27.

…

Neural networks aren’t actually very neural. François Chollet, a deep learning expert at Google, points out in Deep Learning with Python, “There is no evidence that the brain implements anything like the learning mechanisms in use in modern deep learning models.”2 Of course, there’s no reason why machines should mimic the brain; that’s simplistic reverse-anthropomorphic thinking. And, when we see machines showing some semblance of smartness, we anthropomorphize and think that our brains are just some kind of CPU equivalent, cognitive processing units. Deep learning AI is remarkably different from and complementary to human learning.

…

., “Accurate and Reproducible Invasive Breast Cancer Detection in Whole-Slide Images: A Deep Learning Approach for Quantifying Tumor Extent.” Sci Rep, 2017. 7: p. 46450. 51. Ehteshami Bejnordi, B., et al., “Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women with Breast Cancer.” JAMA, 2017. 318(22): pp. 2199–2210. 52. Golden, J. A., “Deep Learning Algorithms for Detection of Lymph Node Metastases from Breast Cancer: Helping Artificial Intelligence Be Seen.” JAMA, 2017. 318(22): pp. 2184–2186. 53. Yang, S. J., et al., “Assessing Microscope Image Focus Quality with Deep Learning.” BMC Bioinformatics, 2018. 19(1): p. 77. 54.

The Deep Learning Revolution (The MIT Press) by Terrence J. Sejnowski

AI winter, Albert Einstein, algorithmic bias, algorithmic trading, AlphaGo, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, augmented reality, autonomous vehicles, backpropagation, Baxter: Rethink Robotics, behavioural economics, bioinformatics, cellular automata, Claude Shannon: information theory, cloud computing, complexity theory, computer vision, conceptual framework, constrained optimization, Conway's Game of Life, correlation does not imply causation, crowdsourcing, Danny Hillis, data science, deep learning, DeepMind, delayed gratification, Demis Hassabis, Dennis Ritchie, discovery of DNA, Donald Trump, Douglas Engelbart, driverless car, Drosophila, Elon Musk, en.wikipedia.org, epigenetics, Flynn Effect, Frank Gehry, future of work, Geoffrey Hinton, Google Glasses, Google X / Alphabet X, Guggenheim Bilbao, Gödel, Escher, Bach, haute couture, Henri Poincaré, I think there is a world market for maybe five computers, industrial robot, informal economy, Internet of things, Isaac Newton, Jim Simons, John Conway, John Markoff, John von Neumann, language acquisition, Large Hadron Collider, machine readable, Mark Zuckerberg, Minecraft, natural language processing, Neil Armstrong, Netflix Prize, Norbert Wiener, OpenAI, orbital mechanics / astrodynamics, PageRank, pattern recognition, pneumatic tube, prediction markets, randomized controlled trial, Recombinant DNA, recommendation engine, Renaissance Technologies, Rodney Brooks, self-driving car, Silicon Valley, Silicon Valley startup, Socratic dialogue, speech recognition, statistical model, Stephen Hawking, Stuart Kauffman, theory of mind, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, traveling salesman, Turing machine, Von Neumann architecture, Watson beat the top human players on Jeopardy!, world market for maybe five computers, X Prize, Yogi Berra

When you get Google to translate for you, it now uses deep learning designed by Dean’s Google 192 Chapter 12 Figure 12.9 NASCAR jacket Terry Sejnowski wore to open the 2015 NIPS Conference in Montreal. Sponsors ranged from top-tier Internet companies to financial and media companies. They all have a stake in deep learning. Courtesy of the NIPS Foundation. Brain team. When you google a search term, deep learning helps to rank the results. When you talk to the Google assistant, it uses deep learning to recognize the words you are saying, and as it gets better at holding a conversation with you, it will be using deep learning to serve you better.

…

Chips 205 Inside Information 219 Consciousness 233 Nature Is Cleverer Than We Are 245 Deep Intelligence 261 169 viii Acknowledgments 269 Recommended Reading 275 Glossary 281 Notes 285 Index 321 Contents Preface P P r r e e f f a a c c e e © Massachusetts Institute of TechnologyAll Rights Reserved If you use voice recognition on an Android phone or Google Translate on the Internet, you have communicated with neural networks1 trained by deep learning. In the last few years, deep learning has generated enough profit for Google to cover the costs of all its futuristic projects at Google X, including self-driving cars, Google Glass, and Google Brain.2 Google was one of the first Internet companies to embrace deep learning; in 2013, it hired Geoffrey Hinton, the father of deep learning, and other companies are racing to catch up. The recent progress in artificial intelligence (AI) was made by reverse engineering brains.

…

All doctors will become better at diagnosing rare skin diseases with the help of deep learning.16 Deep Cancer The detection of metastatic breast cancer in images of lymph node biopsies on slides is done by experts who make mistakes, mistakes that have deadly consequences. This is a pattern recognition problem for which deep learning should excel. And indeed, a deep learning network trained on a large dataset of slides for which ground truth was known reached an accuracy of 0.925, good but not as good as experts who achieved 0.966 on the same test set.17 However, when the predictions of deep learning were combined The Rise of Machine Learning 11 Figure 1.5 Artist’s impression of a deep learning network diagnosing a skin lesion with high accuracy, cover of February 2, 2017, issue of Nature.

Rule of the Robots: How Artificial Intelligence Will Transform Everything by Martin Ford

AI winter, Airbnb, algorithmic bias, algorithmic trading, Alignment Problem, AlphaGo, Amazon Mechanical Turk, Amazon Web Services, artificial general intelligence, Automated Insights, autonomous vehicles, backpropagation, basic income, Big Tech, big-box store, call centre, carbon footprint, Chris Urmson, Claude Shannon: information theory, clean water, cloud computing, commoditize, computer age, computer vision, Computing Machinery and Intelligence, coronavirus, correlation does not imply causation, COVID-19, crowdsourcing, data is the new oil, data science, deep learning, deepfake, DeepMind, Demis Hassabis, deskilling, disruptive innovation, Donald Trump, Elon Musk, factory automation, fake news, fulfillment center, full employment, future of work, general purpose technology, Geoffrey Hinton, George Floyd, gig economy, Gini coefficient, global pandemic, Googley, GPT-3, high-speed rail, hype cycle, ImageNet competition, income inequality, independent contractor, industrial robot, informal economy, information retrieval, Intergovernmental Panel on Climate Change (IPCC), Internet of things, Jeff Bezos, job automation, John Markoff, Kiva Systems, knowledge worker, labor-force participation, Law of Accelerating Returns, license plate recognition, low interest rates, low-wage service sector, Lyft, machine readable, machine translation, Mark Zuckerberg, Mitch Kapor, natural language processing, Nick Bostrom, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, Ocado, OpenAI, opioid epidemic / opioid crisis, passive income, pattern recognition, Peter Thiel, Phillips curve, post scarcity, public intellectual, Ray Kurzweil, recommendation engine, remote working, RFID, ride hailing / ride sharing, Robert Gordon, Rodney Brooks, Rubik’s Cube, Sam Altman, self-driving car, Silicon Valley, Silicon Valley startup, social distancing, SoftBank, South of Market, San Francisco, special economic zone, speech recognition, stealth mode startup, Stephen Hawking, superintelligent machines, TED Talk, The Future of Employment, The Rise and Fall of American Growth, the scientific method, Turing machine, Turing test, Tyler Cowen, Tyler Cowen: Great Stagnation, Uber and Lyft, uber lyft, universal basic income, very high income, warehouse automation, warehouse robotics, Watson beat the top human players on Jeopardy!, WikiLeaks, women in the workforce, Y Combinator

John Markoff, “Scientists see promise in deep-learning programs,” New York Times, November 23, 2012, www.nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html. 10. Dario Amodei and Danny Hernandez, “AI and Compute,” OpenAI Blog, May 16, 2018, openai.com/blog/ai-and-compute/. 11. Will Knight, “Facebook’s head of AI says the field will soon ‘hit the wall,’” Wired, December 4, 2019, www.wired.com/story/facebooks-ai-says-field-hit-wall/. 12. Kim Martineau, “Shrinking deep learning’s carbon footprint,” MIT News, August 7, 2020, news.mit.edu/2020/shrinking-deep-learning-carbon-footprint-0807. 13.

…

While the central processing chip that powers your laptop computer might have two, or perhaps four, computational “cores,” a contemporary high-end GPU would likely have thousands of specialized cores, all of which can crunch numbers at high speed simultaneously. Once researchers discovered that the calculations required by deep learning applications were broadly similar to those needed to render graphics, they began to turn en masse to GPUs, which rapidly evolved into the primary hardware platform for artificial intelligence. Indeed, this transition was a key enabler of the deep learning revolution that took hold beginning in 2012. In September of that year, a team of AI researchers from the University of Toronto put deep learning on the technology industry’s radar by prevailing at the ImageNet Large Scale Visual Recognition Challenge, an important annual event focused on machine vision.

…

We’ll delve further into the history of deep learning in Chapter 4. The University of Toronto’s team used GPUs manufactured by NVIDIA, a company founded in 1993 whose business focused exclusively on designing and manufacturing state-of-the-art graphics chips. In the wake of the 2012 ImageNet competition and the ensuing widespread recognition of the powerful synergy between deep learning and GPUs, the company’s trajectory shifted dramatically, transforming it into one of the most prominent technology companies associated with the rise of artificial intelligence. Evidence of the deep learning revolution manifested directly in the company’s market value: between January 2012 and January 2020 NVIDIA’s shares soared by more than 1,500 percent.

Driverless: Intelligent Cars and the Road Ahead by Hod Lipson, Melba Kurman

AI winter, Air France Flight 447, AlphaGo, Amazon Mechanical Turk, autonomous vehicles, backpropagation, barriers to entry, butterfly effect, carbon footprint, Chris Urmson, cloud computing, computer vision, connected car, creative destruction, crowdsourcing, DARPA: Urban Challenge, deep learning, digital map, Donald Shoup, driverless car, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, General Motors Futurama, Geoffrey Hinton, Google Earth, Google X / Alphabet X, Hans Moravec, high net worth, hive mind, ImageNet competition, income inequality, industrial robot, intermodal, Internet of things, Jeff Hawkins, job automation, Joseph Schumpeter, lone genius, Lyft, megacity, Network effects, New Urbanism, Oculus Rift, pattern recognition, performance metric, Philippa Foot, precision agriculture, RFID, ride hailing / ride sharing, Second Machine Age, self-driving car, Silicon Valley, smart cities, speech recognition, statistical model, Steve Jobs, technoutopianism, TED Talk, Tesla Model S, Travis Kalanick, trolley problem, Uber and Lyft, uber lyft, Unsafe at Any Speed, warehouse robotics

In the past few years, in search of deep-learning expertise, entire divisions of automotive companies have migrated to Silicon Valley. Deep learning is why software giants like Google and Baidu, already armed with expertise in managing huge banks of data and building intelligent software, are giving the once-invincible automotive giants a run for their money. Deep learning has been so revolutionary to the AI community that its reverberations are still unfolding as we write this book and will likely continue to unfold in the years ahead. Cars won’t be the only technology that’s transformed by deep learning. We predict that deep learning will change the developmental trajectory of mobile robotics in general.

…

To encourage third-party developers to build intelligent applications using their software tools, Google, Microsoft, and Facebook have each launched their own version of an open source deep-learning development platform. Despite its usefulness, it has taken decades for deep-learning software to develop. Like other types of machine-learning software, a deep-learning network is data driven, consuming huge amounts of visual data in the form of digital images or videos. Improved performance, enabled by the recent maturation of enabling technologies such as high-speed computers and digital cameras, isn’t the only reason deep-learning software has recently gained acceptance, however. The other reason is political.

…

Inside the network There are several different types of deep-learning networks for image recognition, each with its own twist on the basic architecture and unique refinements in the way the training algorithm is applied. Deep learning is a fast-growing field and new architectures and algorithms appear every few weeks. One common characteristic, however, is that deep-learning networks use a cascade of many layers of artificial neurons to “extract” features from digital images that the software then identifies and labels. Cutting-edge deep-learning networks often have more than 100 layers of artificial neurons (contrast this to Rosenblatt’s Perception, which had only a single layer of eight neurons). Some people believe that deep-learning networks recognize objects the way humans do, by first recognizing a particular tiny feature, then abstracting that to a broader, more abstract concept.

Artificial Intelligence: A Guide for Thinking Humans by Melanie Mitchell

Ada Lovelace, AI winter, Alignment Problem, AlphaGo, Amazon Mechanical Turk, Apple's 1984 Super Bowl advert, artificial general intelligence, autonomous vehicles, backpropagation, Bernie Sanders, Big Tech, Boston Dynamics, Cambridge Analytica, Charles Babbage, Claude Shannon: information theory, cognitive dissonance, computer age, computer vision, Computing Machinery and Intelligence, dark matter, deep learning, DeepMind, Demis Hassabis, Douglas Hofstadter, driverless car, Elon Musk, en.wikipedia.org, folksonomy, Geoffrey Hinton, Gödel, Escher, Bach, I think there is a world market for maybe five computers, ImageNet competition, Jaron Lanier, job automation, John Markoff, John von Neumann, Kevin Kelly, Kickstarter, license plate recognition, machine translation, Mark Zuckerberg, natural language processing, Nick Bostrom, Norbert Wiener, ought to be enough for anybody, paperclip maximiser, pattern recognition, performance metric, RAND corporation, Ray Kurzweil, recommendation engine, ride hailing / ride sharing, Rodney Brooks, self-driving car, sentiment analysis, Silicon Valley, Singularitarianism, Skype, speech recognition, Stephen Hawking, Steve Jobs, Steve Wozniak, Steven Pinker, strong AI, superintelligent machines, tacit knowledge, tail risk, TED Talk, the long tail, theory of mind, There's no reason for any individual to have a computer in his home - Ken Olsen, trolley problem, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, world market for maybe five computers

This acqui-hire instantly put Google at the forefront of deep learning. Soon after, Yann LeCun was lured away from his full-time New York University professorship by Facebook to head up its newly formed AI lab. It didn’t take long before all the big tech companies (as well as many smaller ones) were snapping up deep-learning experts and their graduate students as fast as possible. Seemingly overnight, deep learning became the hottest part of AI, and expertise in deep learning guaranteed computer scientists a large salary in Silicon Valley or, better yet, venture capital funding for their proliferating deep-learning start-up companies.

…

A recent AI survey paper summed it up: “Because we don’t deeply understand intelligence or know how to produce general AI, rather than cutting off any avenues of exploration, to truly make progress we should embrace AI’s ‘anarchy of methods.’”14 But since the 2010s, one family of AI methods—collectively called deep learning (or deep neural networks)—has risen above the anarchy to become the dominant AI paradigm. In fact, in much of the popular media, the term artificial intelligence itself has come to mean “deep learning.” This is an unfortunate inaccuracy, and I need to clarify the distinction. AI is a field that includes a broad set of approaches, with the goal of creating machines with intelligence. Deep learning is only one such approach. Deep learning is itself one method among many in the field of machine learning, a subfield of AI in which machines “learn” from data or from their own “experiences.”

…

.… There’s only a few hundred people in the world that can do that really well.”6 Actually, the number of deep-learning experts is growing quickly; many universities now offer courses in the subject, and a growing list of companies have started their own deep-learning training programs for employees. Membership in the deep-learning club can be quite lucrative. At a recent conference I attended, a leader of Microsoft’s AI product group spoke to the audience about the company’s efforts to hire young deep-learning engineers: “If a kid knows how to train five layers of neural networks, the kid can demand five figures.

AI Superpowers: China, Silicon Valley, and the New World Order by Kai-Fu Lee

"World Economic Forum" Davos, AI winter, Airbnb, Albert Einstein, algorithmic bias, algorithmic trading, Alignment Problem, AlphaGo, artificial general intelligence, autonomous vehicles, barriers to entry, basic income, bike sharing, business cycle, Cambridge Analytica, cloud computing, commoditize, computer vision, corporate social responsibility, cotton gin, creative destruction, crony capitalism, data science, deep learning, DeepMind, Demis Hassabis, Deng Xiaoping, deskilling, Didi Chuxing, Donald Trump, driverless car, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, fake news, full employment, future of work, general purpose technology, Geoffrey Hinton, gig economy, Google Chrome, Hans Moravec, happiness index / gross national happiness, high-speed rail, if you build it, they will come, ImageNet competition, impact investing, income inequality, informal economy, Internet of things, invention of the telegraph, Jeff Bezos, job automation, John Markoff, Kickstarter, knowledge worker, Lean Startup, low skilled workers, Lyft, machine translation, mandatory minimum, Mark Zuckerberg, Menlo Park, minimum viable product, natural language processing, Neil Armstrong, new economy, Nick Bostrom, OpenAI, pattern recognition, pirate software, profit maximization, QR code, Ray Kurzweil, recommendation engine, ride hailing / ride sharing, risk tolerance, Robert Mercer, Rodney Brooks, Rubik’s Cube, Sam Altman, Second Machine Age, self-driving car, sentiment analysis, sharing economy, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, Skype, SoftBank, Solyndra, special economic zone, speech recognition, Stephen Hawking, Steve Jobs, strong AI, TED Talk, The Future of Employment, Travis Kalanick, Uber and Lyft, uber lyft, universal basic income, urban planning, vertical integration, Vision Fund, warehouse robotics, Y Combinator

Those changes include the Chinese AI frenzy that AlphaGo’s matches sparked amid the underlying technology that powered it to victory. AlphaGo runs on deep learning, a groundbreaking approach to artificial intelligence that has turbocharged the cognitive capabilities of machines. Deep-learning-based programs can now do a better job than humans at identifying faces, recognizing speech, and issuing loans. For decades, the artificial intelligence revolution always looked to be five years away. But with the development of deep learning over the past few years, that revolution has finally arrived. It will usher in an era of massive productivity increases but also widespread disruptions in labor markets—and profound sociopsychological effects on people—as artificial intelligence takes over human jobs across all sorts of industries.

…

To understand why, we must first grasp the basics of the technology and how it is set to transform our world. A BRIEF HISTORY OF DEEP LEARNING Machine learning—the umbrella term for the field that includes deep learning—is a history-altering technology but one that is lucky to have survived a tumultuous half-century of research. Ever since its inception, artificial intelligence has undergone a number of boom-and-bust cycles. Periods of great promise have been followed by “AI winters,” when a disappointing lack of practical results led to major cuts in funding. Understanding what makes the arrival of deep learning different requires a quick recap of how we got here. Back in the mid-1950s, the pioneers of artificial intelligence set themselves an impossibly lofty but well-defined mission: to recreate human intelligence in a machine.

…

After decades spent on the margins of AI research, neural networks hit the mainstream overnight, this time in the form of deep learning. That breakthrough promised to thaw the ice from the latest AI winter, and for the first time truly bring AI’s power to bear on a range of real-world problems. Researchers, futurists, and tech CEOs all began buzzing about the massive potential of the field to decipher human speech, translate documents, recognize images, predict consumer behavior, identify fraud, make lending decisions, help robots “see,” and even drive a car. PULLING BACK THE CURTAIN ON DEEP LEARNING So how does deep learning do this? Fundamentally, these algorithms use massive amounts of data from a specific domain to make a decision that optimizes for a desired outcome.

Machine Translation by Thierry Poibeau

Alignment Problem, AlphaGo, AltaVista, augmented reality, call centre, Claude Shannon: information theory, cloud computing, combinatorial explosion, crowdsourcing, deep learning, DeepMind, easy for humans, difficult for computers, en.wikipedia.org, geopolitical risk, Google Glasses, information retrieval, Internet of things, language acquisition, machine readable, machine translation, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, natural language processing, Necker cube, Norbert Wiener, RAND corporation, Robert Mercer, seminal paper, Skype, speech recognition, statistical model, technological singularity, Turing test, wikimedia commons

This explains why it is impossible to manually specify all the information that would be necessary for an automatic machine translation system, but also why the translation task has remained highly challenging and computationally expensive up to the present time. In this context, deep learning provides an interesting approach that seems especially fitted for the challenges involved in improving human language processing. An Overview of Deep Learning for Machine Translation Deep learning achieved its first success in image recognition. Rather than using a group of predefined characteristics, deep learning generally operates from a very large set of examples (hundreds of thousands of images of faces, for example) to automatically extract the most relevant characteristics (called features in machine learning).

…

A very promising trend of research is however to try to get meaningful representations of the internal model calculated by the neural network, so as to better understand how the whole approach works. The deep learning approach to machine translation (or neural machine translation) has proven efficient, first, on short sentences in closely related languages, and more recently on long sentences as well as more diverse languages. Progress is very quick, and the deep learning approach can be considered a revolution for the domain, as was the statistical approach at the beginning of the 1990s. It is interesting to note that deep learning approaches spread very quickly. All the major players in the domain (Google, Bing, Facebook, Systran, etc.) are moving forward to deep learning, and 2016 saw the deployment of the first online systems based on this approach.

…

Deep learning, on the contrary, makes it possible, at least in theory, to learn complex characteristics fully autonomously and gradually from the data, without any prior human effort. In the case of machine translation, deep learning makes it possible to envision systems where very few elements are specified manually, the idea being to let the system infer by itself the best representation from the data. A translation system based solely on deep learning (aka “deep learning machine translation” or “neural machine translation”) thus simply consists of an “encoder” (the part of the system that analyzes the training data) and a “decoder” (the part of the system that automatically produces a translation from a given sentence, based on the data analyzed by the encoder).

Python Data Analytics: With Pandas, NumPy, and Matplotlib by Fabio Nelli

Amazon Web Services, backpropagation, centre right, computer vision, data science, Debian, deep learning, DevOps, functional programming, Google Earth, Guido van Rossum, Internet of things, optical character recognition, pattern recognition, sentiment analysis, speech recognition, statistical model, web application

“Science leads us forward in knowledge, but only analysis makes us more aware” This book is dedicated to all those who are constantly looking for awareness Table of Contents Chapter 1:​ An Introduction to Data Analysis Data Analysis Knowledge Domains of the Data Analyst Computer Science Mathematics and Statistics Machine Learning and Artificial Intelligence Professional Fields of Application Understanding the Nature of the Data When the Data Become Information When the Information Becomes Knowledge Types of Data The Data Analysis Process Problem Definition Data Extraction Data Preparation Data Exploration/​Visualization Predictive Modeling Model Validation Deployment Quantitative and Qualitative Data Analysis Open Data Python and Data Analysis Conclusions Chapter 2:​ Introduction to the Python World Python—The Programming Language Python—The Interpreter Python 2 and Python 3 Installing Python Python Distributions Using Python Writing Python Code IPython PyPI—The Python Package Index The IDEs for Python SciPy NumPy Pandas matplotlib Conclusions Chapter 3:​ The NumPy Library NumPy:​ A Little History The NumPy Installation Ndarray:​ The Heart of the Library Create an Array Types of Data The dtype Option Intrinsic Creation of an Array Basic Operations Arithmetic Operators The Matrix Product Increment and Decrement Operators Universal Functions (ufunc) Aggregate Functions Indexing, Slicing, and Iterating Indexing Slicing Iterating an Array Conditions and Boolean Arrays Shape Manipulation Array Manipulation Joining Arrays Splitting Arrays General Concepts Copies or Views of Objects Vectorization Broadcasting Structured Arrays Reading and Writing Array Data on Files Loading and Saving Data in Binary Files Reading Files with Tabular Data Conclusions Chapter 4:​ The pandas Library—An Introduction pandas:​ The Python Data Analysis Library Installation of pandas Installation from Anaconda Installation from PyPI Installation on Linux Installation from Source A Module Repository for Windows Testing Your pandas Installation Getting Started with pandas Introduction to pandas Data Structures The Series The DataFrame The Index Objects Other Functionalities on Indexes Reindexing Dropping Arithmetic and Data Alignment Operations Between Data Structures Flexible Arithmetic Methods Operations Between DataFrame and Series Function Application and Mapping Functions by Element Functions by Row or Column Statistics Functions Sorting and Ranking Correlation and Covariance “Not a Number” Data Assigning a NaN Value Filtering Out NaN Values Filling in NaN Occurrences Hierarchical Indexing and Leveling Reordering and Sorting Levels Summary Statistic by Level Conclusions Chapter 5:​ pandas:​ Reading and Writing Data I/​O API Tools CSV and Textual Files Reading Data in CSV or Text Files Using RegExp to Parse TXT Files Reading TXT Files Into Parts Writing Data in CSV Reading and Writing HTML Files Writing Data in HTML Reading Data from an HTML File Reading Data from XML Reading and Writing Data on Microsoft Excel Files JSON Data The Format HDF5 Pickle—Python Object Serialization Serialize a Python Object with cPickle Pickling with pandas Interacting with Databases Loading and Writing Data with SQLite3 Loading and Writing Data with PostgreSQL Reading and Writing Data with a NoSQL Database:​ MongoDB Conclusions Chapter 6:​ pandas in Depth:​ Data Manipulation Data Preparation Merging Concatenating Combining Pivoting Removing Data Transformation Removing Duplicates Mapping Discretization and Binning Detecting and Filtering Outliers Permutation Random Sampling String Manipulation Built-in Methods for String Manipulation Regular Expressions Data Aggregation GroupBy A Practical Example Hierarchical Grouping Group Iteration Chain of Transformations Functions on Groups Advanced Data Aggregation Conclusions Chapter 7:​ Data Visualization with matplotlib The matplotlib Library Installation The IPython and IPython QtConsole The matplotlib Architecture Backend Layer Artist Layer Scripting Layer (pyplot) pylab and pyplot pyplot A Simple Interactive Chart The Plotting Window Set the Properties of the Plot matplotlib and NumPy Using the kwargs Working with Multiple Figures and Axes Adding Elements to the Chart Adding Text Adding a Grid Adding a Legend Saving Your Charts Saving the Code Converting Your Session to an HTML File Saving Your Chart Directly as an Image Handling Date Values Chart Typology Line Charts Line Charts with pandas Histograms Bar Charts Horizontal Bar Charts Multiserial Bar Charts Multiseries Bar Charts with pandas Dataframe Multiseries Stacked Bar Charts Stacked Bar Charts with a pandas Dataframe Other Bar Chart Representations Pie Charts Pie Charts with a pandas Dataframe Advanced Charts Contour Plots Polar Charts The mplot3d Toolkit 3D Surfaces Scatter Plots in 3D Bar Charts in 3D Multi-Panel Plots Display Subplots Within Other Subplots Grids of Subplots Conclusions Chapter 8:​ Machine Learning with scikit-learn The scikit-learn Library Machine Learning Supervised and Unsupervised Learning Training Set and Testing Set Supervised Learning with scikit-learn The Iris Flower Dataset The PCA Decomposition K-Nearest Neighbors Classifier Diabetes Dataset Linear Regression:​ The Least Square Regression Support Vector Machines (SVMs) Support Vector Classification (SVC) Nonlinear SVC Plotting Different SVM Classifiers Using the Iris Dataset Support Vector Regression (SVR) Conclusions Chapter 9: Deep Learning with TensorFlow Artificial Intelligence, Machine Learning, and Deep Learning Artificial intelligence Machine Learning Is a Branch of Artificial Intelligence Deep Learning Is a Branch of Machine Learning The Relationship Between Artificial Intelligence, Machine Learning, and Deep Learning Deep Learning Neural Networks and GPUs Data Availability:​ Open Data Source, Internet of Things, and Big Data Python Deep Learning Python Frameworks Artificial Neural Networks How Artificial Neural Networks Are Structured Single Layer Perceptron (SLP) Multi Layer Perceptron (MLP) Correspondence Between Artificial and Biological Neural Networks TensorFlow TensorFlow:​ Google’s Framework TensorFlow:​ Data Flow Graph Start Programming with TensorFlow Installing TensorFlow Programming with the IPython QtConsole The Model and Sessions in TensorFlow Tensors Operation on Tensors Single Layer Perceptron with TensorFlow Before Starting Data To Be Analyzed The SLP Model Definition Learning Phase Test Phase and Accuracy Calculation Multi Layer Perceptron (with One Hidden Layer) with TensorFlow The MLP Model Definition Learning Phase Test Phase and Accuracy Calculation Multi Layer Perceptron (with Two Hidden Layers) with TensorFlow Test Phase and Accuracy Calculation Evaluation of Experimental Data Conclusions Chapter 10:​ An Example— Meteorological Data A Hypothesis to Be Tested:​ The Influence of the Proximity of the Sea The System in the Study:​ The Adriatic Sea and the Po Valley Finding the Data Source Data Analysis on Jupyter Notebook Analysis of Processed Meteorological Data The RoseWind Calculating the Mean Distribution of the Wind Speed Conclusions Chapter 11:​ Embedding the JavaScript D3 Library in the IPython Notebook The Open Data Source for Demographics The JavaScript D3 Library Drawing a Clustered Bar Chart The Choropleth Maps The Choropleth Map of the U.​S.​

…

In this chapter you can have an introductory overview of the world of deep learning, and the artificial neural networks on which its techniques are based. Furthermore, among the new Python frameworks for deep learning, you will use TensorFlow, which is proving to be an excellent tool for research and development of deep learning analysis techniques. With this library you will see how to develop different models of neural networks that are the basis of deep learning. Artificial Intelligence, Machine Learning, and Deep Learning For anyone dealing with the world of data analysis, these three terms are ultimately very common on the web, in text, and on seminars related to the subject.

…

The Relationship Between Artificial Intelligence, Machine Learning, and Deep Learning To sum up, in this section you have seen that machine learning and deep learning are actually subclasses of artificial intelligence. Figure 9-1 shows a schematization of classes in this relationship. Figure 9-1Schematization of the relationship between artificial intelligence, machine learning, and deep learning Deep Learning In this section, you will learn about some significant factors that led to the development of deep learning and read why only in these last years have there been so many steps forward.

AI 2041 by Kai-Fu Lee, Chen Qiufan

3D printing, Abraham Maslow, active measures, airport security, Albert Einstein, AlphaGo, Any sufficiently advanced technology is indistinguishable from magic, artificial general intelligence, augmented reality, autonomous vehicles, basic income, bitcoin, blockchain, blue-collar work, Cambridge Analytica, carbon footprint, Charles Babbage, computer vision, contact tracing, coronavirus, corporate governance, corporate social responsibility, COVID-19, CRISPR, cryptocurrency, DALL-E, data science, deep learning, deepfake, DeepMind, delayed gratification, dematerialisation, digital map, digital rights, digital twin, Elon Musk, fake news, fault tolerance, future of work, Future Shock, game design, general purpose technology, global pandemic, Google Glasses, Google X / Alphabet X, GPT-3, happiness index / gross national happiness, hedonic treadmill, hiring and firing, Hyperloop, information security, Internet of things, iterative process, job automation, language acquisition, low earth orbit, Lyft, Maslow's hierarchy, mass immigration, mirror neurons, money: store of value / unit of account / medium of exchange, mutually assured destruction, natural language processing, Neil Armstrong, Nelson Mandela, OpenAI, optical character recognition, pattern recognition, plutocrats, post scarcity, profit motive, QR code, quantitative easing, Richard Feynman, ride hailing / ride sharing, robotic process automation, Satoshi Nakamoto, self-driving car, seminal paper, Silicon Valley, smart cities, smart contracts, smart transportation, Snapchat, social distancing, speech recognition, Stephen Hawking, synthetic biology, telemarketer, Tesla Model S, The future is already here, trolley problem, Turing test, uber lyft, universal basic income, warehouse automation, warehouse robotics, zero-sum game

“The Golden Elephant” explores deep learning’s stunning potential—as well as its potential pitfalls, like perpetuating bias. So how do researchers develop, train, and use deep learning? What are its limitations? How is deep learning fueled by data? Why are Internet and finance the two most promising initial industries for AI? In what conditions does deep learning optimally work? When it does work, why does it seem to work so well? And what are the downsides and pitfalls of AI? WHAT IS DEEP LEARNING? Inspired by the tangled webs of neurons in our brains, deep learning constructs software layers of artificial neural networks with input and output layers.

…

That question about trade-offs lies at the heart of “The Golden Elephant,” which introduces the foundational AI concept of deep leaning. Deep learning is a recent AI breakthrough. Among the many subfields of AI, machine learning is the field that has produced the most successful applications, and within machine learning, the biggest advance is “deep learning”—so much so that the terms “AI,” “machine learning,” and “deep learning” are sometimes used interchangeably (if imprecisely). Deep learning supercharged excitement in AI in 2016 when it powered AlphaGo’s stunning victory over a human competitor in Go, Asia’s most popular intellectual board game. After that headline-grabbing turn, deep learning became a prominent part of most commercial AI applications, and it is featured in most of the stories in AI 2041.

…

The figure on this page shows such a “cat recognition” deep learning neural network. During this process, deep learning is mathematically trained to maximize the value of an “objective function.” In the case of cat recognition, the objective function is the probability of correct recognitions of “cat” vs. “no cat.” Once “trained,” this deep learning network is essentially a giant mathematical equation that can be tested on images it hasn’t seen, and it will perform “inference” to determine the presence or absence of cats. The advent of deep learning pushed AI capabilities from unusable to usable for many domains.

Applied Artificial Intelligence: A Handbook for Business Leaders by Mariya Yao, Adelyn Zhou, Marlene Jia

Airbnb, algorithmic bias, AlphaGo, Amazon Web Services, artificial general intelligence, autonomous vehicles, backpropagation, business intelligence, business process, call centre, chief data officer, cognitive load, computer vision, conceptual framework, data science, deep learning, DeepMind, en.wikipedia.org, fake news, future of work, Geoffrey Hinton, industrial robot, information security, Internet of things, iterative process, Jeff Bezos, job automation, machine translation, Marc Andreessen, natural language processing, new economy, OpenAI, pattern recognition, performance metric, price discrimination, randomized controlled trial, recommendation engine, robotic process automation, Salesforce, self-driving car, sentiment analysis, Silicon Valley, single source of truth, skunkworks, software is eating the world, source of truth, sparse data, speech recognition, statistical model, strong AI, subscription business, technological singularity, The future is already here

You can also find updated technical resources on our book website, appliedaibook.com. Deep Learning Deep learning is a subfield of machine learning that builds algorithms by using multi-layered artificial neural networks, which are mathematical structures loosely inspired by how biological neurons fire. Neural networks were invented in the 1950s, but recent advances in computational power and algorithm design—as well as the growth of big data—have enabled deep learning algorithms to approach human-level performance in tasks such as speech recognition and image classification. Deep learning, in combination with reinforcement learning, enabled Google DeepMind’s AlphaGo to defeat human world champions of Go in 2016, a feat that many experts had considered to be computationally impossible.

…

Much media attention has been focused on deep learning, and an increasing number of sophisticated technology companies have successfully implemented deep learning for enterprise-scale products. Google replaced previous statistical methods for machine translation with neural networks to achieve superior performance.(4) Microsoft announced in 2017 that they had achieved human parity in conversational speech recognition.(5) Promising computer vision startups like Clarifai employ deep learning to achieve state-of-the-art results in recognizing objects in images and video for Fortune 500 brands.(6) While deep learning models outperform older machine learning approaches to many problems, they are more difficult to develop because they require robust training of data sets and specialized expertise in optimization techniques.

…

Operationalizing and productizing models for enterprise-scale usage also requires different but equally difficult-to-acquire technical expertise. In practice, using simpler AI approaches like older, non-deep-learning machine learning techniques can produce faster and better results than fancy neural nets can. Rather than building custom deep learning solutions, many enterprises opt for Machine Learning as a Service (MLaaS) solutions from Google, Amazon, IBM, Microsoft, or leading AI startups. Deep learning also suffers from technical drawbacks. Successful models typically require a large volume of reliably-labeled data, which enterprises often lack.

Four Battlegrounds by Paul Scharre

2021 United States Capitol attack, 3D printing, active measures, activist lawyer, AI winter, AlphaGo, amateurs talk tactics, professionals talk logistics, artificial general intelligence, ASML, augmented reality, Automated Insights, autonomous vehicles, barriers to entry, Berlin Wall, Big Tech, bitcoin, Black Lives Matter, Boeing 737 MAX, Boris Johnson, Brexit referendum, business continuity plan, business process, carbon footprint, chief data officer, Citizen Lab, clean water, cloud computing, commoditize, computer vision, coronavirus, COVID-19, crisis actor, crowdsourcing, DALL-E, data is not the new oil, data is the new oil, data science, deep learning, deepfake, DeepMind, Demis Hassabis, Deng Xiaoping, digital map, digital rights, disinformation, Donald Trump, drone strike, dual-use technology, Elon Musk, en.wikipedia.org, endowment effect, fake news, Francis Fukuyama: the end of history, future of journalism, future of work, game design, general purpose technology, Geoffrey Hinton, geopolitical risk, George Floyd, global supply chain, GPT-3, Great Leap Forward, hive mind, hustle culture, ImageNet competition, immigration reform, income per capita, interchangeable parts, Internet Archive, Internet of things, iterative process, Jeff Bezos, job automation, Kevin Kelly, Kevin Roose, large language model, lockdown, Mark Zuckerberg, military-industrial complex, move fast and break things, Nate Silver, natural language processing, new economy, Nick Bostrom, one-China policy, Open Library, OpenAI, PalmPilot, Parler "social media", pattern recognition, phenotype, post-truth, purchasing power parity, QAnon, QR code, race to the bottom, RAND corporation, recommendation engine, reshoring, ride hailing / ride sharing, robotic process automation, Rodney Brooks, Rubik’s Cube, self-driving car, Shoshana Zuboff, side project, Silicon Valley, slashdot, smart cities, smart meter, Snapchat, social software, sorting algorithm, South China Sea, sparse data, speech recognition, Steve Bannon, Steven Levy, Stuxnet, supply-chain attack, surveillance capitalism, systems thinking, tech worker, techlash, telemarketer, The Brussels Effect, The Signal and the Noise by Nate Silver, TikTok, trade route, TSMC

They had huge resources for managing the nontechnical aspects of government contracting. With just two total employees, Deep Learning Analytics had none. However, Deep Learning Analytics did have better technology. “We didn’t know anything about radar,” John said, but “it turned out not to be as important as some of the other things” such as “knowing how to do deep learning well” and “a really disciplined software engineering approach.” He said, “That’s one of the recurring stories of . . . deep learning since 2012, is that domain expertise isn’t always the thing that’s going to matter so much. . . . The labeled data was sufficient.” Deep Learning Analytics won a $6 million contract from DARPA for the TRACE program, beating out competitors that had better human expert knowledge on radar imaging.

…

Many of the datasets used to train deep neural networks are massive. ImageNet, the image database that kicked off the deep learning revolution in 2012, includes 14 million images. In order for a neural network to learn what an object looks like, such as a “cat,” “car,” or “chair,” it needs many examples to develop an internal representation of that object. For any given object, ImageNet contains roughly 500 to 1,000 images of that object to allow for a rich set of examples. Deep learning is a more data-intensive process than writing a set of rules for behavior, but deep learning can also be vastly more effective at building intelligent systems for some tasks.

…

While AI models are often trained at large data centers, the lower compute requirements mean that inference can increasingly be done on edge devices, such as smartphones, IoT devices, intelligent video cameras, or autonomous cars. Both training and inference are done on computer chips, and advances in computing hardware has been fundamental to the deep learning revolution. Graphics processing units (GPUs) have emerged as a key enabler for deep learning because of their ability to do parallel computation (which is valuable for neural networks) better than traditional central processing units (CPUs). A McKinsey study estimated that 97 percent of deep learning training in data centers in 2017 used GPUs. As machine learning researchers have turned to training bigger models on ever-larger datasets, they have also needed increasingly massive amounts of compute.

Possible Minds: Twenty-Five Ways of Looking at AI by John Brockman

AI winter, airport security, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Alignment Problem, AlphaGo, artificial general intelligence, Asilomar, autonomous vehicles, basic income, Benoit Mandelbrot, Bill Joy: nanobots, Bletchley Park, Buckminster Fuller, cellular automata, Claude Shannon: information theory, Computing Machinery and Intelligence, CRISPR, Daniel Kahneman / Amos Tversky, Danny Hillis, data science, David Graeber, deep learning, DeepMind, Demis Hassabis, easy for humans, difficult for computers, Elon Musk, Eratosthenes, Ernest Rutherford, fake news, finite state, friendly AI, future of work, Geoffrey Hinton, Geoffrey West, Santa Fe Institute, gig economy, Hans Moravec, heat death of the universe, hype cycle, income inequality, industrial robot, information retrieval, invention of writing, it is difficult to get a man to understand something, when his salary depends on his not understanding it, James Watt: steam engine, Jeff Hawkins, Johannes Kepler, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John von Neumann, Kevin Kelly, Kickstarter, Laplace demon, Large Hadron Collider, Loebner Prize, machine translation, market fundamentalism, Marshall McLuhan, Menlo Park, military-industrial complex, mirror neurons, Nick Bostrom, Norbert Wiener, OpenAI, optical character recognition, paperclip maximiser, pattern recognition, personalized medicine, Picturephone, profit maximization, profit motive, public intellectual, quantum cryptography, RAND corporation, random walk, Ray Kurzweil, Recombinant DNA, Richard Feynman, Rodney Brooks, self-driving car, sexual politics, Silicon Valley, Skype, social graph, speech recognition, statistical model, Stephen Hawking, Steven Pinker, Stewart Brand, strong AI, superintelligent machines, supervolcano, synthetic biology, systems thinking, technological determinism, technological singularity, technoutopianism, TED Talk, telemarketer, telerobotics, The future is already here, the long tail, the scientific method, theory of mind, trolley problem, Turing machine, Turing test, universal basic income, Upton Sinclair, Von Neumann architecture, Whole Earth Catalog, Y2K, you are the product, zero-sum game

Within a few years, Judea’s Bayesian networks had completely overshadowed the previous rule-based approaches to artificial intelligence. The advent of deep learning—in which computers, in effect, teach themselves to be smarter by observing tons of data—has given him pause, because this method lacks transparency. While recognizing the impressive achievements in deep learning by colleagues such as Michael I. Jordan and Geoffrey Hinton, he feels uncomfortable with this kind of opacity. He set out to understand the theoretical limitations of deep-learning systems and points out that basic barriers exist that will prevent them from achieving a human kind of intelligence, no matter what we do.

…

The new excitement about AI comes because AI researchers have recently produced powerful and effective versions of both of these learning methods. But there is nothing profoundly new about the methods themselves. BOTTOM-UP DEEP LEARNING In the 1980s, computer scientists devised an ingenious way to get computers to detect patterns in data: connectionist, or neural-network, architecture (the “neural” part was, and still is, metaphorical). The approach fell into the doldrums in the 1990s but has recently been revived with powerful “deep-learning” methods like Google’s DeepMind. For example, you can give a deep-learning program a bunch of Internet images labeled “cat,” others labeled “house,” and so on. The program can detect the patterns differentiating the two sets of images and use that information to label new images correctly.

…

For example, researchers at Google’s DeepMind used a combination of deep learning and reinforcement learning to teach a computer to play Atari video games. The computer knew nothing about how the games worked. It began by acting randomly and got information only about what the screen looked like at each moment and how well it had scored. Deep learning helped interpret the features on the screen, and reinforcement learning rewarded the system for higher scores. The computer got very good at playing several of the games, but it also completely bombed on others that were just as easy for humans to master. A similar combination of deep learning and reinforcement learning has enabled the success of DeepMind’s AlphaZero, a program that managed to beat human players at both chess and Go, equipped with only a basic knowledge of the rules of the game and some planning capacities.

The Myth of Artificial Intelligence: Why Computers Can't Think the Way We Do by Erik J. Larson

AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, Alignment Problem, AlphaGo, Amazon Mechanical Turk, artificial general intelligence, autonomous vehicles, Big Tech, Black Swan, Bletchley Park, Boeing 737 MAX, business intelligence, Charles Babbage, Claude Shannon: information theory, Computing Machinery and Intelligence, conceptual framework, correlation does not imply causation, data science, deep learning, DeepMind, driverless car, Elon Musk, Ernest Rutherford, Filter Bubble, Geoffrey Hinton, Georg Cantor, Higgs boson, hive mind, ImageNet competition, information retrieval, invention of the printing press, invention of the wheel, Isaac Newton, Jaron Lanier, Jeff Hawkins, John von Neumann, Kevin Kelly, Large Hadron Collider, Law of Accelerating Returns, Lewis Mumford, Loebner Prize, machine readable, machine translation, Nate Silver, natural language processing, Nick Bostrom, Norbert Wiener, PageRank, PalmPilot, paperclip maximiser, pattern recognition, Peter Thiel, public intellectual, Ray Kurzweil, retrograde motion, self-driving car, semantic web, Silicon Valley, social intelligence, speech recognition, statistical model, Stephen Hawking, superintelligent machines, tacit knowledge, technological singularity, TED Talk, The Coming Technological Singularity, the long tail, the scientific method, The Signal and the Noise by Nate Silver, The Wisdom of Crowds, theory of mind, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, Yochai Benkler

So-­called adversarial attacks are not unique to AlexNet, ­either. Deep learning systems showing impressive per­for­mance on image recognition in fact do not understand what they are perceiving. It is therefore easy to expose the brittleness of the approach. Other experiments have drastically degraded per­for­mance by simply including background objects, easily ignored by ­humans, but problematic for deep learning systems. In other experiments, images that look like salt-­and-­pepper static on TVs—­random assemblages of black and 166 T he P rob­lem of I nference white pixels—­fool deep learning systems, which might classify them as pictures of armadillos, cheetahs, or centipedes.

…

The system produced question-­answer pairs, accumulating evidence in the pro­cessing pipeline, and scored the list of pairs using statistical I nference and L anguage I I 231 techniques—­a ll pos­si­ble ­because of data about past games, where the outcome is known. It’s notable that Watson d­ idn’t use deep learning—at least not the version of Watson that outplayed ­human champions in the televised event in 2011. Deep learning would not have helped—­and ­here again, this is a nod to the ingenuity of the IBM team. A relatively s­ imple machine-­learning technique known as regularized logistic regression was used, even though more power­ful learning algorithms ­were available. (Deep learning in 2011 was still relatively unknown.) More power­ful learning systems would simply incur more computational training and testing expense—­A I is a toolkit, in the end.

…

Classical AI scientists dismissed t­ hese as “shallow” or “empirical,” ­because statistical approaches using data ­d idn’t use knowledge and c­ ouldn’t h­ andle reasoning or planning very well (if at all). But with the web providing the much-­needed data, the approaches started showing promise. The deep learning “revolution” began around 2006, with early work by Geoff Hinton, Yann LeCun, and Yoshua Bengio. By 2010, Google, Microsoft, and other Big Tech companies w ­ ere using neural networks for major consumer applications such as voice recognition, and by 2012, Android smartphones featured neural network technology. From about this time up through 2020 (as I write this), deep learning has been the hammer causing all the prob­lems of AI to look like a nail—­ prob­lems that can be approached “from the ground up,” like playing games and recognizing voice and image data, now account for most of the research and commercial dollars in AI.

The Road to Conscious Machines by Michael Wooldridge

Ada Lovelace, AI winter, algorithmic bias, AlphaGo, Andrew Wiles, Anthropocene, artificial general intelligence, Asilomar, augmented reality, autonomous vehicles, backpropagation, basic income, Bletchley Park, Boeing 747, British Empire, call centre, Charles Babbage, combinatorial explosion, computer vision, Computing Machinery and Intelligence, DARPA: Urban Challenge, deep learning, deepfake, DeepMind, Demis Hassabis, don't be evil, Donald Trump, driverless car, Elaine Herzberg, Elon Musk, Eratosthenes, factory automation, fake news, future of work, gamification, general purpose technology, Geoffrey Hinton, gig economy, Google Glasses, intangible asset, James Watt: steam engine, job automation, John von Neumann, Loebner Prize, Minecraft, Mustafa Suleyman, Nash equilibrium, Nick Bostrom, Norbert Wiener, NP-complete, P = NP, P vs NP, paperclip maximiser, pattern recognition, Philippa Foot, RAND corporation, Ray Kurzweil, Rodney Brooks, self-driving car, Silicon Valley, Stephen Hawking, Steven Pinker, strong AI, technological singularity, telemarketer, Tesla Model S, The Coming Technological Singularity, The Future of Employment, the scientific method, theory of mind, Thomas Bayes, Thomas Kuhn: the structure of scientific revolutions, traveling salesman, trolley problem, Turing machine, Turing test, universal basic income, Von Neumann architecture, warehouse robotics

But it is not obvious that just continuing to refine deep learning techniques will address this problem. Deep learning will be part of the solution, but a proper solution will, I think, require something much more than just a larger neural net, or more processing power, or more training data in the form of boring French novels. It will require breakthroughs at least as dramatic as deep learning itself. I suspect those breakthroughs will require explicitly represented knowledge as well as deep learning: somehow, we will have to bridge the gap between the world of explicitly represented knowledge, and the world of deep learning and neural nets.

…

Like the story of AI itself, the story of neural networks is a troubled one: there have been two ‘neural net winters’, and as recently as the turn of the century, many in AI regarded neural networks as a dead or dying field. But neural nets ultimately triumphed, and the new idea driving their resurgence is a technique called deep learning. Deep learning is the core technology of DeepMind. I will tell you the DeepMind story, and how the systems that DeepMind built attracted global adulation. But while deep learning is a powerful and important technique, it isn’t the end of the story for AI, so, just as we did with other AI technologies, we’ll discuss its limitations in detail too. Machine Learning, Briefly The goal of machine learning is to have programs that can compute a desired output from a given input, without being given an explicit recipe for how to do this.

…

But a resurgence did indeed begin, around the year 2006, and it led to the biggest and most highly publicized expansion ever in the history of AI. The big idea that drove the third wave of neural net research went by the name of deep learning.6 I would love to tell you that there was a single key idea which characterizes deep learning, but in truth the term refers to a collection of related ideas. Deep learning means deep in at least three different senses. Of these, perhaps the most important, as the name suggests, is simply the idea of having more layers. Each layer can process a problem at a different level of abstraction – layers close to the input layer handle low-level concepts in the data (such as the edges in a picture), and as we move deeper into the network, we find more abstract concepts being handled.

Thinking Machines: The Inside Story of Artificial Intelligence and Our Race to Build the Future by Luke Dormehl

"World Economic Forum" Davos, Ada Lovelace, agricultural Revolution, AI winter, Albert Einstein, Alexey Pajitnov wrote Tetris, algorithmic management, algorithmic trading, AlphaGo, Amazon Mechanical Turk, Apple II, artificial general intelligence, Automated Insights, autonomous vehicles, backpropagation, Bletchley Park, book scanning, borderless world, call centre, cellular automata, Charles Babbage, Claude Shannon: information theory, cloud computing, computer vision, Computing Machinery and Intelligence, correlation does not imply causation, crowdsourcing, deep learning, DeepMind, driverless car, drone strike, Elon Musk, Flash crash, Ford Model T, friendly AI, game design, Geoffrey Hinton, global village, Google X / Alphabet X, Hans Moravec, hive mind, industrial robot, information retrieval, Internet of things, iterative process, Jaron Lanier, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John von Neumann, Kickstarter, Kodak vs Instagram, Law of Accelerating Returns, life extension, Loebner Prize, machine translation, Marc Andreessen, Mark Zuckerberg, Menlo Park, Mustafa Suleyman, natural language processing, Nick Bostrom, Norbert Wiener, out of africa, PageRank, paperclip maximiser, pattern recognition, radical life extension, Ray Kurzweil, recommendation engine, remote working, RFID, scientific management, self-driving car, Silicon Valley, Skype, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, social intelligence, speech recognition, Stephen Hawking, Steve Jobs, Steve Wozniak, Steven Pinker, strong AI, superintelligent machines, tech billionaire, technological singularity, The Coming Technological Singularity, The Future of Employment, Tim Cook: Apple, Tony Fadell, too big to fail, traumatic brain injury, Turing machine, Turing test, Vernor Vinge, warehouse robotics, Watson beat the top human players on Jeopardy!

Having a computer that knows what a cat is may not sound like a particularly useful achievement, but the ability to use deep learning for computer vision has a host of real-world uses. One startup called Dextro is using deep learning to create better tools for online video searches. Instead of relying on keyword tags, Dextro’s neural nets scans through live videos, analysing both audio and image. Ask it about David Cameron, for example, and it will bring up not just Conservative Party videos, but also video in which the UK Prime Minister is only mentioned in passing. Facebook, meanwhile, uses deep learning to automatically tag images. In June 2014, the social network published a paper describing what it refers to as its ‘DeepFace’ facial recognition technology.

…

In June 2014, the social network published a paper describing what it refers to as its ‘DeepFace’ facial recognition technology. Thanks to deep learning, Facebook’s algorithms have proven almost as accurate as the human brain when it comes to looking at two photos and saying whether they show the same person, regardless of whether different lighting or camera angles are used. Facebook is also using deep learning to create technology able to describe images to blind users – such as verbalising the fact that an image shows a particular friend riding a bicycle through the English countryside on a summer’s day. Other projects combine deep learning with robotics. One group of researchers from the University of Maryland has taught a robot how to cook a simple meal by simply showing it ‘how-to’ cooking videos available on YouTube.

…

Nobody, not even the smartest neural network, can be expected to learn what something is if they are never explicitly told. In fact, what Hinton discovered was that unsupervised learning could be used to train up layers of features, one layer at a time. This was the catalyst in the field of ‘deep learning’, currently the hottest area in AI. You can think of a deep learning network a bit like a factory line. After the raw materials are input, they are passed down the conveyor belt, with each subsequent stop or layer extracting a different set of high-level features. To continue the example of an image recognition network, the first layer may be used to analyse pixel brightness.

Python for Algorithmic Trading: From Idea to Cloud Deployment by Yves Hilpisch

algorithmic trading, Amazon Web Services, automated trading system, backtesting, barriers to entry, bitcoin, Brownian motion, cloud computing, coronavirus, cryptocurrency, data science, deep learning, Edward Thorp, fiat currency, global macro, Gordon Gekko, Guido van Rossum, implied volatility, information retrieval, margin call, market microstructure, Myron Scholes, natural language processing, paper trading, passive investing, popular electronics, prediction markets, quantitative trading / quantitative finance, random walk, risk free rate, risk/return, Rubik’s Cube, seminal paper, Sharpe ratio, short selling, sorting algorithm, systematic trading, transaction costs, value at risk

For example, assume that a stock trades 10 USD under its 200 days SMA level of 100. It is then expected that the stock price will return to its SMA level sometime soon. Machine and Deep Learning With machine and deep learning algorithms, one generally takes a more black box approach to predicting market movements. For simplicity and reproducibility, the examples in this book mainly rely on historical return observations as features to train machine and deep learning algorithms to predict stock market movements. This book does not introduce algorithmic trading in a systematic fashion. Since the focus lies on applying Python in this fascinating field, readers not familiar with algorithmic trading should consult dedicated resources on the topic, some of which are cited in this chapter and the chapters that follow.

…

With this reasoning, the prediction problem basically boils down to a classification problem of deciding whether there will be an upwards or downwards movement. Different machine learning algorithms have been developed to attack such classification problems. This chapter introduces logistic regression, as a typical baseline algorithm, for classification. Deep learning-based strategies Deep learning has been popularized by such technological giants as Facebook. Similar to machine learning algorithms, deep learning algorithms based on neural networks allow one to attack classification problems faced in financial market prediction. The chapter is organized as follows. “Using Linear Regression for Market Movement Prediction” introduces linear regression as a technique to predict index levels and the direction of price movements.

…

For example, testing the very same strategy instead of in-sample on an out-of-sample data set and adding transaction costs—as two ways of getting to a more realistic picture—often shows that the performance of the considered strategy “suddenly” trails the base instrument performance-wise or turns to a net loss. Using Deep Learning for Market Movement Prediction Right from the open sourcing and publication by Google, the deep learning library TensorFlow has attracted much interest and wide-spread application. This section applies TensorFlow in the same way that the previous section applied scikit-learn to the prediction of stock market movements modeled as a classification problem. However, TensorFlow is not used directly; it is rather used via the equally popular Keras deep learning package. Keras can be thought of as providing a higher level abstraction to the TensorFlow package with an easier to understand and use API.

Succeeding With AI: How to Make AI Work for Your Business by Veljko Krunic

AI winter, Albert Einstein, algorithmic trading, AlphaGo, Amazon Web Services, anti-fragile, anti-pattern, artificial general intelligence, autonomous vehicles, Bayesian statistics, bioinformatics, Black Swan, Boeing 737 MAX, business process, cloud computing, commoditize, computer vision, correlation coefficient, data is the new oil, data science, deep learning, DeepMind, en.wikipedia.org, fail fast, Gini coefficient, high net worth, information retrieval, Internet of things, iterative process, job automation, Lean Startup, license plate recognition, minimum viable product, natural language processing, recommendation engine, self-driving car, sentiment analysis, Silicon Valley, six sigma, smart cities, speech recognition, statistical model, strong AI, tail risk, The Design of Experiments, the scientific method, web application, zero-sum game

During discussions, the team has mentioned that this problem could be solved using either an SVM, a decision tree, logistic regression, or a deep learning-based classification. Should you use deep learning? After all, it’s an exceedingly popular technology, has a substantial mindshare, and could solve the problem. Or should you use one of the other suggested options? Answer to question 5:  You can use a deep learning-based classifier, but I typically wouldn’t try it as my first (or even second) choice. Unless your survey is a monster with a thousand questions, it’s unclear that you’ll be able to train a large deep learning network at all.  I’m not persuaded that for the typical survey of only a few questions, more complicated methods are going to produce better results.

…

During discussions, the team has mentioned that this problem could be solved using either an SVM, a decision tree, logistic regression, or a deep learning-based classification. Should you use deep learning? After all, it’s an exceedingly popular technology, has a substantial mindshare, and could solve the problem. Or should you use one of the other suggested options? Question 6: You answered question 5 using an algorithm of your choice. Suppose the algorithm you chose didn’t provide a good enough prediction of a customer returning the product. Should you use a better ML algorithm? Is it now time to use the latest and greatest from the field of deep learning? Summary  Every AI project uses some form of the ML pipeline.

…

Harris, Murphy, and Vaisman’s book [66] provides a good summary of the state of data science before the advancement of deep learning. Data scientist—A practitioner of the field of data science. Many sources (including this book) classify AI practitioners as data scientists. Database administrator (DBA)—A professional responsible for the maintenance of a database. Most commonly, a DBA would be responsible for maintaining a RDBMS-based database. Deep learning—A subfield of AI that uses artificial neural networks arranged in a significant number of layers. In the last few years, deep learning algorithms have been successful in a large number of highly visible applications, including image processing and speech and audio recognition.

Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell

3D printing, Ada Lovelace, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Alfred Russel Wallace, algorithmic bias, AlphaGo, Andrew Wiles, artificial general intelligence, Asilomar, Asilomar Conference on Recombinant DNA, augmented reality, autonomous vehicles, basic income, behavioural economics, Bletchley Park, blockchain, Boston Dynamics, brain emulation, Cass Sunstein, Charles Babbage, Claude Shannon: information theory, complexity theory, computer vision, Computing Machinery and Intelligence, connected car, CRISPR, crowdsourcing, Daniel Kahneman / Amos Tversky, data science, deep learning, deepfake, DeepMind, delayed gratification, Demis Hassabis, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Ernest Rutherford, fake news, Flash crash, full employment, future of work, Garrett Hardin, Geoffrey Hinton, Gerolamo Cardano, Goodhart's law, Hans Moravec, ImageNet competition, Intergovernmental Panel on Climate Change (IPCC), Internet of things, invention of the wheel, job automation, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John Nash: game theory, John von Neumann, Kenneth Arrow, Kevin Kelly, Law of Accelerating Returns, luminiferous ether, machine readable, machine translation, Mark Zuckerberg, multi-armed bandit, Nash equilibrium, Nick Bostrom, Norbert Wiener, NP-complete, OpenAI, openstreetmap, P = NP, paperclip maximiser, Pareto efficiency, Paul Samuelson, Pierre-Simon Laplace, positional goods, probability theory / Blaise Pascal / Pierre de Fermat, profit maximization, RAND corporation, random walk, Ray Kurzweil, Recombinant DNA, recommendation engine, RFID, Richard Thaler, ride hailing / ride sharing, Robert Shiller, robotic process automation, Rodney Brooks, Second Machine Age, self-driving car, Shoshana Zuboff, Silicon Valley, smart cities, smart contracts, social intelligence, speech recognition, Stephen Hawking, Steven Pinker, superintelligent machines, surveillance capitalism, Thales of Miletus, The Future of Employment, The Theory of the Leisure Class by Thorstein Veblen, Thomas Bayes, Thorstein Veblen, Tragedy of the Commons, transport as a service, trolley problem, Turing machine, Turing test, universal basic income, uranium enrichment, vertical integration, Von Neumann architecture, Wall-E, warehouse robotics, Watson beat the top human players on Jeopardy!, web application, zero-sum game

The current champion learning algorithm for machine translation is a form of so-called deep learning, and it produces a rule in the form of an artificial neural network with hundreds of layers and millions of parameters.D Other deep learning algorithms have turned out to be very good at classifying the objects in images and recognizing the words in a speech signal. Machine translation, speech recognition, and visual object recognition are three of the most important subfields in AI, which is why there has been so much excitement about the prospects for deep learning. One can argue almost endlessly about whether deep learning will lead directly to human-level AI.

…

News article on Geoff Hinton having second thoughts about deep networks: Steve LeVine, “Artificial intelligence pioneer says we need to start over,” Axios, September 15, 2017. 9. A catalog of shortcomings of deep learning: Gary Marcus, “Deep learning: A critical appraisal,” arXiv:1801.00631 (2018). 10. A popular textbook on deep learning, with a frank assessment of its weaknesses: François Chollet, Deep Learning with Python (Manning Publications, 2017). 11. An explanation of explanation-based learning: Thomas Dietterich, “Learning at the knowledge level,” Machine Learning 1 (1986): 287–315. 12. A superficially quite different explanation of explanation-based learning: John Laird, Paul Rosenbloom, and Allen Newell, “Chunking in Soar: The anatomy of a general learning mechanism,” Machine Learning 1 (1986): 11–46.

…

Computers are also made of circuits, both in their memories and in their processing units; but those circuits have to be arranged in certain ways, and layers of software have to be added, before the computer can support the operation of high-level programming languages and logical reasoning systems. At present, however, there is no sign that deep learning systems can develop such capabilities by themselves—nor does it make scientific sense to require them to do so. There are further reasons to think that deep learning may reach a plateau well short of general intelligence, but it’s not my purpose here to diagnose all the problems: others, both inside8 and outside9 the deep learning community, have noted many of them. The point is that simply creating larger and deeper networks and larger data sets and bigger machines is not enough to create human-level AI.

Coders: The Making of a New Tribe and the Remaking of the World by Clive Thompson

"Margaret Hamilton" Apollo, "Susan Fowler" uber, 2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, 4chan, 8-hour work day, Aaron Swartz, Ada Lovelace, AI winter, air gap, Airbnb, algorithmic bias, AlphaGo, Amazon Web Services, Andy Rubin, Asperger Syndrome, augmented reality, Ayatollah Khomeini, backpropagation, barriers to entry, basic income, behavioural economics, Bernie Sanders, Big Tech, bitcoin, Bletchley Park, blockchain, blue-collar work, Brewster Kahle, Brian Krebs, Broken windows theory, call centre, Cambridge Analytica, cellular automata, Charles Babbage, Chelsea Manning, Citizen Lab, clean water, cloud computing, cognitive dissonance, computer vision, Conway's Game of Life, crisis actor, crowdsourcing, cryptocurrency, Danny Hillis, data science, David Heinemeier Hansson, deep learning, DeepMind, Demis Hassabis, disinformation, don't be evil, don't repeat yourself, Donald Trump, driverless car, dumpster diving, Edward Snowden, Elon Musk, Erik Brynjolfsson, Ernest Rutherford, Ethereum, ethereum blockchain, fake news, false flag, Firefox, Frederick Winslow Taylor, Free Software Foundation, Gabriella Coleman, game design, Geoffrey Hinton, glass ceiling, Golden Gate Park, Google Hangouts, Google X / Alphabet X, Grace Hopper, growth hacking, Guido van Rossum, Hacker Ethic, hockey-stick growth, HyperCard, Ian Bogost, illegal immigration, ImageNet competition, information security, Internet Archive, Internet of things, Jane Jacobs, John Markoff, Jony Ive, Julian Assange, Ken Thompson, Kickstarter, Larry Wall, lone genius, Lyft, Marc Andreessen, Mark Shuttleworth, Mark Zuckerberg, Max Levchin, Menlo Park, meritocracy, microdosing, microservices, Minecraft, move 37, move fast and break things, Nate Silver, Network effects, neurotypical, Nicholas Carr, Nick Bostrom, no silver bullet, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, Oculus Rift, off-the-grid, OpenAI, operational security, opioid epidemic / opioid crisis, PageRank, PalmPilot, paperclip maximiser, pattern recognition, Paul Graham, paypal mafia, Peter Thiel, pink-collar, planetary scale, profit motive, ransomware, recommendation engine, Richard Stallman, ride hailing / ride sharing, Rubik’s Cube, Ruby on Rails, Sam Altman, Satoshi Nakamoto, Saturday Night Live, scientific management, self-driving car, side project, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, single-payer health, Skype, smart contracts, Snapchat, social software, software is eating the world, sorting algorithm, South of Market, San Francisco, speech recognition, Steve Wozniak, Steven Levy, systems thinking, TaskRabbit, tech worker, techlash, TED Talk, the High Line, Travis Kalanick, Uber and Lyft, Uber for X, uber lyft, universal basic income, urban planning, Wall-E, Watson beat the top human players on Jeopardy!, WeWork, WikiLeaks, women in the workforce, Y Combinator, Zimmermann PGP, éminence grise

You could now create neural nets with many layers, or even dozens: “deep learning,” as it’s called, because of how many layers are stacked up. By 2012, the field had a seismic breakthrough. Up at the University of Toronto, the British computer scientist Geoff Hinton had been beavering away for two decades on improving neural networks. That year he and a team of students showed off the most impressive neural net yet—by soundly beating competitors at an annual AI shootout. The ImageNet challenge, as it’s known, is an annual competition among AI researchers to see whose system is best at recognizing images. That year, Hinton’s deep-learning neural net got only 15.3 percent of the images wrong.

…

At first, the result surprised them, though on reflection it made sense: There are a lot of cats on YouTube, so any self-learning algorithm told to pick out salient features that recur over and over again might, in essence, discover humanity’s online obsession with felines. Still, it was a spookily humanlike bit of reasoning. The Terminator was coming to life, and it could grasp the concept of cats! Google soon began throwing enormous resources at deep learning, developing its abilities and integrating it into as many products as possible. They trained deep-learning nets on language pairings—showing it, say, all the Canadian parliamentary proceedings that were translated into both English and French, or their own collections of crowdsourced translations. When they were done, Google Translate became, in a single night, remarkably better—so much improved that Japanese scholars were marveling at the machine’s deft ability to translate complex literary passages between their language and English.

…

When they were done, Google Translate became, in a single night, remarkably better—so much improved that Japanese scholars were marveling at the machine’s deft ability to translate complex literary passages between their language and English. A few short years later, deep learning had swept the world of software. Companies everywhere were rushing to incorporate it into their services. Ng was snapped up by Baidu, the Chinese search giant, as it frantically sought to catch up to Google’s AI wave. Facebook engineers had long been using many different styles of machine learning to help recognize faces in photos, filter stories in the News Feed, and predict whether users would click on an ad; it set up an experimental AI research lab, and soon Facebook was producing a deep-learning model that could recognize faces with 97.35 percent accuracy, 27 percent better than the state of the art (“closely approaching human-level performance,” as they noted.)

I, Warbot: The Dawn of Artificially Intelligent Conflict by Kenneth Payne

Abraham Maslow, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, AlphaGo, anti-communist, Any sufficiently advanced technology is indistinguishable from magic, artificial general intelligence, Asperger Syndrome, augmented reality, Automated Insights, autonomous vehicles, backpropagation, Black Lives Matter, Bletchley Park, Boston Dynamics, classic study, combinatorial explosion, computer age, computer vision, Computing Machinery and Intelligence, coronavirus, COVID-19, CRISPR, cuban missile crisis, data science, deep learning, deepfake, DeepMind, delayed gratification, Demis Hassabis, disinformation, driverless car, drone strike, dual-use technology, Elon Musk, functional programming, Geoffrey Hinton, Google X / Alphabet X, Internet of things, job automation, John Nash: game theory, John von Neumann, Kickstarter, language acquisition, loss aversion, machine translation, military-industrial complex, move 37, mutually assured destruction, Nash equilibrium, natural language processing, Nick Bostrom, Norbert Wiener, nuclear taboo, nuclear winter, OpenAI, paperclip maximiser, pattern recognition, RAND corporation, ransomware, risk tolerance, Ronald Reagan, self-driving car, semantic web, side project, Silicon Valley, South China Sea, speech recognition, Stanislav Petrov, stem cell, Stephen Hawking, Steve Jobs, strong AI, Stuxnet, technological determinism, TED Talk, theory of mind, TikTok, Turing machine, Turing test, uranium enrichment, urban sprawl, V2 rocket, Von Neumann architecture, Wall-E, zero-sum game

They can even create wholly convincing photographs of people who’ve never actually existed. Artificial neuroscience The third factor in the deep learning revival was more arcane, but critical. This was the increasing technical sophistication of the neural networks that underpinned connectionism. One important development was the discovery of ‘backprop’, or backward propagation. This was a key bit of maths that allowed the artificial neurons in the connectionist AI to learn effectively. With multiple layers in the modern ‘deep learning network’, and with many more neurons and connections between them, working out the optimum connections between them had been fiendishly difficult.

…

In 2017, the Pentagon stood up an ‘algorithmic warfare cross functional team’, known as Project Maven. The team would consolidate ‘all initiatives that develop, employ, or field artificial intelligence, automation, machine learning, deep learning, and computer vision algorithms’.12 It was a small team, initially, but would grow rapidly. And one of its main contractors on image recognition? Google, of course. The stage was set for the arrival of deep warbots. Hype or hope? The deep learning revolution has produced a new wave of AI hype, only some of which is justified. In rapid succession, connectionist AI delivered results that would have amazed earlier students of symbolic logic.

…

While autonomous weapons and AI systems have been around for more than half a century, the recent surge in deep learning AI creates the possibility of swarming behaviours, and this more than any other feature, will likely have an impact on warbot tactics. The swarm attacks! Swarms don’t require a particularly sophisticated intelligence. At least not sophisticated in the sense we often use the term—carefully weighing goals and ways before deciding. In nature, swarming is often an instinctive behaviour. In robotics it’s the sort of skilful control challenge that deep learning AI excels at. Above the deserts of the US Navy’s vast China Lake research facility, Perdix and the Defense Department recently demonstrated the state of the art.

Surviving AI: The Promise and Peril of Artificial Intelligence by Calum Chace

3D printing, Ada Lovelace, AI winter, Airbnb, Alvin Toffler, artificial general intelligence, augmented reality, barriers to entry, basic income, bitcoin, Bletchley Park, blockchain, brain emulation, Buckminster Fuller, Charles Babbage, cloud computing, computer age, computer vision, correlation does not imply causation, credit crunch, cryptocurrency, cuban missile crisis, deep learning, DeepMind, dematerialisation, Demis Hassabis, discovery of the americas, disintermediation, don't be evil, driverless car, Elon Musk, en.wikipedia.org, epigenetics, Erik Brynjolfsson, everywhere but in the productivity statistics, Flash crash, friendly AI, Geoffrey Hinton, Google Glasses, hedonic treadmill, hype cycle, industrial robot, Internet of things, invention of agriculture, job automation, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John von Neumann, Kevin Kelly, life extension, low skilled workers, machine translation, Mahatma Gandhi, means of production, mutually assured destruction, Neil Armstrong, Nicholas Carr, Nick Bostrom, paperclip maximiser, pattern recognition, peer-to-peer, peer-to-peer model, Peter Thiel, radical life extension, Ray Kurzweil, Robert Solow, Rodney Brooks, Second Machine Age, self-driving car, Silicon Valley, Silicon Valley ideology, Skype, South Sea Bubble, speech recognition, Stanislav Petrov, Stephen Hawking, Steve Jobs, strong AI, technological singularity, TED Talk, The future is already here, The Future of Employment, theory of mind, Turing machine, Turing test, universal basic income, Vernor Vinge, wage slave, Wall-E, zero-sum game

They are particularly useful in speech recognition and handwriting recognition systems. Deep learning Deep learning is a subset of machine learning. Its algorithms use several layers of processing, each taking data from previous layers and passing an output up to the next layer. The nature of the output may vary according to the nature of the input, which is not necessarily binary, just on or off, but can be weighted. The number of layers can vary too, with anything above ten layers seen as very deep learning. Artificial neural nets (ANN) are an important type of deep learning system – indeed some people argue that deep learning is simply a re-branding of neural networks.

…

Early hopes for the quick development of thinking machines were dashed, however, and neural nets fell into disuse until the late 1980s, when they experienced a renaissance along with what came to be known as deep learning thanks to pioneers Yann LeCun (now at Facebook), Geoff Hinton (now at Google) and Yoshua Bengio, a professor at the University of Montreal. Yann LeCun describes deep learning as follows. “A pattern recognition system is like a black box with a camera at one end, a green light and a red light on top, and a whole bunch of knobs on the front. The learning algorithm tries to adjust the knobs so that when, say, a dog is in front of the camera, the red light turns on, and when a car is put in front of the camera, the green light turns on.

…

The speculation that a system containing enough of the types of operations involved in machine learning might generate a conscious mind intrigues some neuroscientists, and strikes others as wildly implausible, or as something that is many years away. Gary Marcus, a psychology professor at New York University, says “deep learning is only part of the larger challenge of building intelligent machines. Such techniques [are] still a long way from integrating abstract knowledge, such as information about what objects are, what they are for, and how they are typically used. The most powerful A.I. systems, like Watson, . . . use techniques like deep learning as just one element in a very complicated ensemble of techniques . . .” (33) Andrew Ng, formerly head of the Google Brain project and now in charge of Baidu’s AI activities, says that current machine learning techniques are like a “cartoon version” of the human brain.

The Economic Singularity: Artificial Intelligence and the Death of Capitalism by Calum Chace

"World Economic Forum" Davos, 3D printing, additive manufacturing, agricultural Revolution, AI winter, Airbnb, AlphaGo, Alvin Toffler, Amazon Robotics, Andy Rubin, artificial general intelligence, augmented reality, autonomous vehicles, banking crisis, basic income, Baxter: Rethink Robotics, Berlin Wall, Bernie Sanders, bitcoin, blockchain, Boston Dynamics, bread and circuses, call centre, Chris Urmson, congestion charging, credit crunch, David Ricardo: comparative advantage, deep learning, DeepMind, Demis Hassabis, digital divide, Douglas Engelbart, Dr. Strangelove, driverless car, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Fairchild Semiconductor, Flynn Effect, full employment, future of work, Future Shock, gender pay gap, Geoffrey Hinton, gig economy, Google Glasses, Google X / Alphabet X, Hans Moravec, Herman Kahn, hype cycle, ImageNet competition, income inequality, industrial robot, Internet of things, invention of the telephone, invisible hand, James Watt: steam engine, Jaron Lanier, Jeff Bezos, job automation, John Markoff, John Maynard Keynes: technological unemployment, John von Neumann, Kevin Kelly, Kiva Systems, knowledge worker, lifelogging, lump of labour, Lyft, machine translation, Marc Andreessen, Mark Zuckerberg, Martin Wolf, McJob, means of production, Milgram experiment, Narrative Science, natural language processing, Neil Armstrong, new economy, Nick Bostrom, Occupy movement, Oculus Rift, OpenAI, PageRank, pattern recognition, post scarcity, post-industrial society, post-work, precariat, prediction markets, QWERTY keyboard, railway mania, RAND corporation, Ray Kurzweil, RFID, Rodney Brooks, Sam Altman, Satoshi Nakamoto, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Skype, SoftBank, software is eating the world, speech recognition, Stephen Hawking, Steve Jobs, TaskRabbit, technological singularity, TED Talk, The future is already here, The Future of Employment, Thomas Malthus, transaction costs, Two Sigma, Tyler Cowen, Tyler Cowen: Great Stagnation, Uber for X, uber lyft, universal basic income, Vernor Vinge, warehouse automation, warehouse robotics, working-age population, Y Combinator, young professional

The way that some game-playing AIs become superhuman in their field is by playing millions of games against versions of themselves and learning from the outcomes.) In deep learning, the algorithms operate in several layers, each layer processing data from previous ones and passing the output up to the next layer. The output is not necessarily binary, just on or off: it can be weighted. The number of layers can vary too, with anything above ten layers seen as very deep learning – although in December 2015 a Microsoft team won the ImageNet competition with a system which employed a massive 152 layers.[lxvi] Deep learning, and especially artificial neural nets (ANNs), are in many ways a return to an older approach to AI which was explored in the 1960s but abandoned because it proved ineffective.

…

“Then you teach it Mandarin: it learns Mandarin, but it also becomes better at English, and quite frankly none of us know exactly why.”[xciii] In December 2015, Baidu announced that its speech recognition system Deep Speech 2 performed better than humans with short phrases out of context.[xciv] It uses deep learning techniques to recognise Mandarin. Learning and innovating It can no longer be said that machines do not learn, or that they cannot invent. In December 2013, DeepMind demonstrated an AI system which used a deep learning technique called unsupervised learning to teach itself to play old-style Atari video games like Breakout and Pong.[xcv] These are games which previous AI systems found hard to play because they involve hand-to-eye co-ordination.

…

doi=10.1257/jep.29.3.3 [lvi] https://reason.com/archives/2015/03/03/how-to-survive-a-robot-uprisin [lvii] http://www.politico.com/magazine/story/2013/11/the-robots-are-here-098995 [lviii] http://www.forbes.com/sites/danschawbel/2015/08/04/geoff-colvin-why-humans-will-triumph-over-machines/2/ [lix] http://www.eastoftheweb.com/short-stories/UBooks/BoyCri.shtml [lx] German academic Marcus Hutter, and Shane Legg, co-founder of DeepMind [lxi] http://www.savethechimps.org/about-us/chimp-facts/ [lxii] The Shape of Automation for Men and Management by Herbert Simon, 1965 [lxiii] Computation: Finite and Infinite Machines by Marvin Minsky, 1967 [lxiv] http://www.wired.com/2016/01/microsoft-neural-net-shows-deep-learning-can-get-way-deeper/ [lxv] http://www.etymonline.com/index.php?term=algorithm [lxvi] http://www.wired.com/2016/01/microsoft-neural-net-shows-deep-learning-can-get-way-deeper/ [lxvii] Moravec wrote about this phenomenon in his 1988 book “Mind Children”. A possible explanation is that the sensory motor skills and spatial awareness that we develop as children are the product of millions of years of evolution.

The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity by Amy Webb

"Friedman doctrine" OR "shareholder theory", Ada Lovelace, AI winter, air gap, Airbnb, airport security, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, algorithmic bias, AlphaGo, Andy Rubin, artificial general intelligence, Asilomar, autonomous vehicles, backpropagation, Bayesian statistics, behavioural economics, Bernie Sanders, Big Tech, bioinformatics, Black Lives Matter, blockchain, Bretton Woods, business intelligence, Cambridge Analytica, Cass Sunstein, Charles Babbage, Claude Shannon: information theory, cloud computing, cognitive bias, complexity theory, computer vision, Computing Machinery and Intelligence, CRISPR, cross-border payments, crowdsourcing, cryptocurrency, Daniel Kahneman / Amos Tversky, data science, deep learning, DeepMind, Demis Hassabis, Deng Xiaoping, disinformation, distributed ledger, don't be evil, Donald Trump, Elon Musk, fail fast, fake news, Filter Bubble, Flynn Effect, Geoffrey Hinton, gig economy, Google Glasses, Grace Hopper, Gödel, Escher, Bach, Herman Kahn, high-speed rail, Inbox Zero, Internet of things, Jacques de Vaucanson, Jeff Bezos, Joan Didion, job automation, John von Neumann, knowledge worker, Lyft, machine translation, Mark Zuckerberg, Menlo Park, move fast and break things, Mustafa Suleyman, natural language processing, New Urbanism, Nick Bostrom, one-China policy, optical character recognition, packet switching, paperclip maximiser, pattern recognition, personalized medicine, RAND corporation, Ray Kurzweil, Recombinant DNA, ride hailing / ride sharing, Rodney Brooks, Rubik’s Cube, Salesforce, Sand Hill Road, Second Machine Age, self-driving car, seminal paper, SETI@home, side project, Silicon Valley, Silicon Valley startup, skunkworks, Skype, smart cities, South China Sea, sovereign wealth fund, speech recognition, Stephen Hawking, strong AI, superintelligent machines, surveillance capitalism, technological singularity, The Coming Technological Singularity, the long tail, theory of mind, Tim Cook: Apple, trade route, Turing machine, Turing test, uber lyft, Von Neumann architecture, Watson beat the top human players on Jeopardy!, zero day

By 2009, Hinton’s lab had applied deep neural nets for speech recognition, and a chance meeting with a Microsoft researcher named Li Deng meant that the technology could be piloted in a meaningful way. Deng, a Chinese deep-learning specialist, was a pioneer in speech recognition using large-scale deep learning. By 2010, the technique was being tested at Google. Just two years later, deep neural nets were being used in commercial products. If you used Google Voice and its transcription services, that was deep learning, and the technique became the basis for all the digital assistants we use today. Siri, Google, and Amazon’s Alexa are all powered by deep learning. The AI community of interdisciplinary researchers had grown significantly since the Dartmouth summer.

…

We can’t evolve significantly on our own, and the existing evolutionary timeframe doesn’t suit our current technological aspirations. The promise of deep learning was an acceleration of the evolution of intelligence itself, which would only temporarily involve humans. A deep neural net would be given a basic set of parameters about the data by a person, and then the system would go out and learn on its own by recognizing patterns using many layers of processing. For researchers, the attraction of deep learning is that by design, machines make decisions unpredictably. Thinking in ways we humans have never imagined—or been able to do ourselves—is vitally important when trying to solve big problems for which there haven’t ever been clear solutions.

…

With all those simulated neurons and layers, exactly what happened and in which order can’t be easily reverse-engineered. One team of Google researchers did try to develop a new technique to make AI more transparent. In essence, the researchers ran a deep-learning image recognition algorithm in reverse to observe how the system recognized certain things such as trees, snails, and pigs. The project, called DeepDream, used a network created by MIT’s Computer Science and AI Lab and ran Google’s deep-learning algorithm in reverse. Instead of training it to recognize objects using the layer-by-layer approach—to learn that a rose is a rose, and a daffodil is a daffodil—instead it was trained to warp the images and generate objects that weren’t there.

AIQ: How People and Machines Are Smarter Together by Nick Polson, James Scott

Abraham Wald, Air France Flight 447, Albert Einstein, algorithmic bias, Amazon Web Services, Atul Gawande, autonomous vehicles, availability heuristic, basic income, Bayesian statistics, Big Tech, Black Lives Matter, Bletchley Park, business cycle, Cepheid variable, Checklist Manifesto, cloud computing, combinatorial explosion, computer age, computer vision, Daniel Kahneman / Amos Tversky, data science, deep learning, DeepMind, Donald Trump, Douglas Hofstadter, Edward Charles Pickering, Elon Musk, epigenetics, fake news, Flash crash, Grace Hopper, Gödel, Escher, Bach, Hans Moravec, Harvard Computers: women astronomers, Higgs boson, index fund, information security, Isaac Newton, John von Neumann, late fees, low earth orbit, Lyft, machine translation, Magellanic Cloud, mass incarceration, Moneyball by Michael Lewis explains big data, Moravec's paradox, more computing power than Apollo, natural language processing, Netflix Prize, North Sea oil, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, p-value, pattern recognition, Pierre-Simon Laplace, ransomware, recommendation engine, Ronald Reagan, Salesforce, self-driving car, sentiment analysis, side project, Silicon Valley, Skype, smart cities, speech recognition, statistical model, survivorship bias, systems thinking, the scientific method, Thomas Bayes, Uber for X, uber lyft, universal basic income, Watson beat the top human players on Jeopardy!, young professional

See also Lee Bell, “Nvidia to Train 100,000 Developers in ‘Deep Learning’ AI to Bolster Healthcare Research,” Forbes.com, May 11, 2017, https://blogs.nvidia.com/blog/2016/12/07/mass-general-researchers-ai/. See also Lee Bell, “Nvidia to Train 100,000 Developers in ‘Deep Learning’ AI to Bolster Healthcare Research,” Forbes.com, May 11, 2017, https://blogs.nvidia.com/blog/2016/12/07/mass-general-researchers-ai/. See also Lee Bell, “Nvidia to Train 100,000 Developers in ‘Deep Learning’ AI to Bolster Healthcare Research,” Forbes.com, May 11, 2017, https://www.forbes.com/sites/leebelltech/2017/05/11/nvidia-to-train-100000-developers-in-deep-learning-ai-to-bolster-health-care-research/. 50.  See, e.g., Tom Simonite, “The Recipe for the Perfect Robot Surgeon,” MIT Technology Review, October 14, 2016, https://www.technologyreview.com/s/602595/the-recipe-for-the-perfect-robot-surgeon/. 51.  

…

But then Koike realized that he could use a piece of open-source AI software from Google, called TensorFlow, to accomplish the same task, by coding up a “deep-learning” algorithm that could classify a cucumber based on a photograph. Koike had never used AI or TensorFlow before, but with all the free resources out there, he didn’t find it hard to teach himself how. When a video of his AI-powered sorting machine hit YouTube, Koike became an international deep-learning/cucumber celebrity. It wasn’t merely that he had given people a feel-good story, saving his mother from hours of drudgery. He’d also sent an inspiring message to students and coders across the world: that if AI can solve problems in cucumber farming, it can solve problems just about anywhere.

…

We’re omitting a lot of details here that are essential for making this strategy succeed, but they’re all just minutiae, the kind of thing you learn if you study AI in graduate school. If you just think “trial and error,” you’re 90% of the way there. Factor 4: Deep Learning In addition to the richness of our models, the size of our data sets, and the speed of our computers, there’s a fourth major way in which prediction rules have improved dramatically: people have learned how to extract useful information from vastly more complicated inputs. If you’ve heard the term “deep learning” and wondered what it means, we’re about to explain. We said at the beginning of the chapter that computers are agnostic about the type of input you give them.

Prediction Machines: The Simple Economics of Artificial Intelligence by Ajay Agrawal, Joshua Gans, Avi Goldfarb

Abraham Wald, Ada Lovelace, AI winter, Air France Flight 447, Airbus A320, algorithmic bias, AlphaGo, Amazon Picking Challenge, artificial general intelligence, autonomous vehicles, backpropagation, basic income, Bayesian statistics, Black Swan, blockchain, call centre, Capital in the Twenty-First Century by Thomas Piketty, Captain Sullenberger Hudson, carbon tax, Charles Babbage, classic study, collateralized debt obligation, computer age, creative destruction, Daniel Kahneman / Amos Tversky, data acquisition, data is the new oil, data science, deep learning, DeepMind, deskilling, disruptive innovation, driverless car, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, everywhere but in the productivity statistics, financial engineering, fulfillment center, general purpose technology, Geoffrey Hinton, Google Glasses, high net worth, ImageNet competition, income inequality, information retrieval, inventory management, invisible hand, Jeff Hawkins, job automation, John Markoff, Joseph Schumpeter, Kevin Kelly, Lyft, Minecraft, Mitch Kapor, Moneyball by Michael Lewis explains big data, Nate Silver, new economy, Nick Bostrom, On the Economy of Machinery and Manufactures, OpenAI, paperclip maximiser, pattern recognition, performance metric, profit maximization, QWERTY keyboard, race to the bottom, randomized controlled trial, Ray Kurzweil, ride hailing / ride sharing, Robert Solow, Salesforce, Second Machine Age, self-driving car, shareholder value, Silicon Valley, statistical model, Stephen Hawking, Steve Jobs, Steve Jurvetson, Steven Levy, strong AI, The Future of Employment, the long tail, The Signal and the Noise by Nate Silver, Tim Cook: Apple, trolley problem, Turing test, Uber and Lyft, uber lyft, US Airways Flight 1549, Vernor Vinge, vertical integration, warehouse automation, warehouse robotics, Watson beat the top human players on Jeopardy!, William Langewiesche, Y Combinator, zero-sum game

The vertical axis measures the error rate, so lower is better. In 2010, the best machine predictions made mistakes in 28 percent of images. In 2012, the contestants used deep learning for the first time and the error rate plunged to 16 percent. As Princeton professor and computer scientist Olga Russakovsky notes, “2012 was really the year when there was a massive breakthrough in accuracy, but it was also a proof of concept for deep learning models, which had been around for decades.”8 Rapid improvements in the algorithms continued, and a team beat the human benchmark in the competition for the first time in 2015.

…

“Mastercard Rolls Out Artificial Intelligence across Its Global Network,” Mastercard press release, November 30, 2016, https://newsroom.mastercard.com/press-releases/mastercard-rolls-out-artificial-intelligence-across-its-global-network/. 3. Adam Geitgey, “Machine Learning Is Fun, Part 5: Language Translation with Deep Learning and the Magic of Sequences,” Medium, August 21, 2016, https://medium.com/@ageitgey/machine-learning-is-fun-part-5-language-translation-with-deep-learning-and-the-magic-of-sequences-2ace0acca0aa. 4. Yiting Sun, “Why 500 Million People in China Are Talking to This AI,”MIT Technology Review, September 14, 2017, https://www.technologyreview.com/s/608841/why-500-million-people-in-china-are-talking-to-this-ai/. 5.

…

Tests That Were Found to Discriminate,” New York Times, July 23, 2009, https://cityroom.blogs.nytimes.com/2009/07/23/the-fire-dept-tests-that-were-found-to-discriminate/?mcubz=0&_r=0; US v. City of New York (FDNY), https://www.justice.gov/archives/crt-fdny/overview. 6. Paul Voosen, “How AI Detectives Are Cracking Open the Black Box of Deep Learning,” Science, July 6, 2017, http://www.sciencemag.org/news/2017/07/how-ai-detectives-are-cracking-open-black-box-deep-learning. 7. T. Blake, C. Nosko, and S. Tadelis, “Consumer Heterogeneity and Paid Search Effectiveness: A Large-Scale Field Experiment,” Econometrica 83 (2015): 155–174. 8. Hossein Hosseini, Baicen Xiao, and Radha Poovendran, “Deceiving Google’s Cloud Video Intelligence API Built for Summarizing Videos” (paper presented at CVPR Workshops, March 31, 2017), https://arxiv.org/pdf/1703.09793.pdf; see also “Artificial Intelligence Used by Google to Scan Videos Could Easily Be Tricked by a Picture of Noodles,” Quartz, April 4, 2017, https://qz.com/948870/the-ai-used-by-google-to-scan-videos-could-easily-be-tricked-by-a-picture-of-noodles/. 9.

The Alignment Problem: Machine Learning and Human Values by Brian Christian

Albert Einstein, algorithmic bias, Alignment Problem, AlphaGo, Amazon Mechanical Turk, artificial general intelligence, augmented reality, autonomous vehicles, backpropagation, butterfly effect, Cambridge Analytica, Cass Sunstein, Claude Shannon: information theory, computer vision, Computing Machinery and Intelligence, data science, deep learning, DeepMind, Donald Knuth, Douglas Hofstadter, effective altruism, Elaine Herzberg, Elon Musk, Frances Oldham Kelsey, game design, gamification, Geoffrey Hinton, Goodhart's law, Google Chrome, Google Glasses, Google X / Alphabet X, Gödel, Escher, Bach, Hans Moravec, hedonic treadmill, ImageNet competition, industrial robot, Internet Archive, John von Neumann, Joi Ito, Kenneth Arrow, language acquisition, longitudinal study, machine translation, mandatory minimum, mass incarceration, multi-armed bandit, natural language processing, Nick Bostrom, Norbert Wiener, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, OpenAI, Panopticon Jeremy Bentham, pattern recognition, Peter Singer: altruism, Peter Thiel, precautionary principle, premature optimization, RAND corporation, recommendation engine, Richard Feynman, Rodney Brooks, Saturday Night Live, selection bias, self-driving car, seminal paper, side project, Silicon Valley, Skinner box, sparse data, speech recognition, Stanislav Petrov, statistical model, Steve Jobs, strong AI, the map is not the territory, theory of mind, Tim Cook: Apple, W. E. B. Du Bois, Wayback Machine, zero-sum game

., “Weight Uncertainty in Neural Networks”; and Hernández-Lobato and Adams, “Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks.” For a more detailed history of these ideas, see Gal, “Uncertainty in Deep Learning.” For an overview of probabilistic methods in machine learning more generally, see Ghahramani, “Probabilistic Machine Learning and Artificial Intelligence.” 17. Yarin Gal, personal interview, July 11, 2019. 18. Yarin Gal, “Modern Deep Learning Through Bayesian Eyes” (lecture), Microsoft Research, December 11, 2015, https://www.microsoft.com/en-us/research/video/modern-deep-learning-through-bayesian-eyes/. 19. For a look at using dropout-ensemble uncertainty to detect adversarial examples, see Smith and Gal, “Understanding Measures of Uncertainty for Adversarial Example Detection.” 20.

…

But on this matter, too, Bellemare would soon come around. Simply plugging deep learning into a classic RL algorithm and running it on seven of the Atari games, Mnih was able to beat every previous RL benchmark in six of them. Not only that: in three of the games, their program appeared to be as good as a human player. They submitted a workshop paper in late 2013, marking their progress.6 “It was just sort of a proof-of-concept paper,” says Bellemare, “that convolutional nets could do this.” “Really,” he says, “it was bringing the deep-learning part to solve what reinforcement-learning researchers hadn’t been able to do in ages, which is to generate these features on the fly.

…

In better coming to understand our own motivations and drives, we then, in turn, have a chance for complementary and reciprocal insights about how to build an artificial intelligence as flexible, resilient, and intellectually omnivorous as our own. Deepak Pathak looks at the success of deep learning and sees one glaring weakness: each system—be it for machine translation, or object recognition, or even game playing—is purpose-built. Training a huge neural network on a heap of manually labeled images was, as we have seen, the paradigm in which deep learning first truly showed its promise. Explicitly drilling a system to categorize images made a system that could categorize images. Fair enough, he says. “But the problem is, these artificial intelligence systems are not actually intelligent.

Ghost Road: Beyond the Driverless Car by Anthony M. Townsend

A Pattern Language, active measures, AI winter, algorithmic trading, Alvin Toffler, Amazon Robotics, asset-backed security, augmented reality, autonomous vehicles, backpropagation, big-box store, bike sharing, Blitzscaling, Boston Dynamics, business process, Captain Sullenberger Hudson, car-free, carbon footprint, carbon tax, circular economy, company town, computer vision, conceptual framework, congestion charging, congestion pricing, connected car, creative destruction, crew resource management, crowdsourcing, DARPA: Urban Challenge, data is the new oil, Dean Kamen, deep learning, deepfake, deindustrialization, delayed gratification, deliberate practice, dematerialisation, deskilling, Didi Chuxing, drive until you qualify, driverless car, drop ship, Edward Glaeser, Elaine Herzberg, Elon Musk, en.wikipedia.org, extreme commuting, financial engineering, financial innovation, Flash crash, food desert, Ford Model T, fulfillment center, Future Shock, General Motors Futurama, gig economy, Google bus, Greyball, haute couture, helicopter parent, independent contractor, inventory management, invisible hand, Jane Jacobs, Jeff Bezos, Jevons paradox, jitney, job automation, John Markoff, John von Neumann, Joseph Schumpeter, Kickstarter, Kiva Systems, Lewis Mumford, loss aversion, Lyft, Masayoshi Son, megacity, microapartment, minimum viable product, mortgage debt, New Urbanism, Nick Bostrom, North Sea oil, Ocado, openstreetmap, pattern recognition, Peter Calthorpe, random walk, Ray Kurzweil, Ray Oldenburg, rent-seeking, ride hailing / ride sharing, Rodney Brooks, self-driving car, sharing economy, Shoshana Zuboff, Sidewalk Labs, Silicon Valley, Silicon Valley startup, Skype, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, SoftBank, software as a service, sovereign wealth fund, Stephen Hawking, Steve Jobs, surveillance capitalism, technological singularity, TED Talk, Tesla Model S, The Coming Technological Singularity, The Death and Life of Great American Cities, The future is already here, The Future of Employment, The Great Good Place, too big to fail, traffic fines, transit-oriented development, Travis Kalanick, Uber and Lyft, uber lyft, urban planning, urban sprawl, US Airways Flight 1549, Vernor Vinge, vertical integration, Vision Fund, warehouse automation, warehouse robotics

“Contemporary neural networks do well on challenges that remain close to their core training data,” writes New York University computer scientist Gary Marcus in an exhaustive 2018 critique of deep learning, “but start to break down on cases further out in the periphery.” Nowhere are the limits of deep learning becoming clearer than in the development of self-driving vehicles. The most catastrophic failures involving automated driving so far have all occurred around so-called edge cases, those unexpected events where data to train deep-learning models was insufficient or simply nonexistent—a pedestrian walking a bicycle across a darkened street midblock (Tempe, Arizona, March 18, 2018); a white truck trailer occluded against a brightly lit sky (Williston, Florida, May 7, 2016); an unusual set of road-surface markings at a highway off-ramp (Mountain View, California, March 23, 2018).

…

Chips like Pegasus are shifting the center of effort inside your car, from powertrain to CPU, and changing the kinds of “fuel” needed. The motor in your old car harnessed the power of internal combustion. It sucked in gasoline and transformed it into mechanical power to move you down the road. This new motor in your AV is powered by deep learning. It ingests gigabytes of data and spits out a stream of insights to guide you on your way. Deep learning sounds more mysterious than it is. The artificial neural networks that make it work were first invented more than 70 years ago. These algorithms, loosely based on mammalian brains, were the basis of a promising early branch of AI research. But after several highprofile failures, the mainstream research community largely abandoned the approach.

…

Even more remarkable was their seemingly intuitive power (learning). You didn’t have to program a deep learning model with descriptions of exactly what to look for to, say, identify photographs of cats. All you had to do was wind the mechanism up with a million pictures of cats and it could deduce the fundamental indicators of cat-ness all by itself. This process, called “training,” works by slowly calibrating the nodes within and between the stack’s various layers, strengthening the connections that contribute to accurate results and pruning those that don’t. Deep learning does have at least one enormous drawback, however. It is a ravenous consumer of computer power.

Talk to Me: How Voice Computing Will Transform the Way We Live, Work, and Think by James Vlahos

Albert Einstein, AltaVista, Amazon Mechanical Turk, Amazon Web Services, augmented reality, Automated Insights, autonomous vehicles, backpropagation, Big Tech, Cambridge Analytica, Chuck Templeton: OpenTable:, cloud computing, Colossal Cave Adventure, computer age, deep learning, DeepMind, Donald Trump, Elon Musk, fake news, Geoffrey Hinton, information retrieval, Internet of things, Jacques de Vaucanson, Jeff Bezos, lateral thinking, Loebner Prize, machine readable, machine translation, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, Mark Zuckerberg, Menlo Park, natural language processing, Neal Stephenson, Neil Armstrong, OpenAI, PageRank, pattern recognition, Ponzi scheme, randomized controlled trial, Ray Kurzweil, Ronald Reagan, Rubik’s Cube, self-driving car, sentiment analysis, Silicon Valley, Skype, Snapchat, speech recognition, statistical model, Steve Jobs, Steve Wozniak, Steven Levy, TechCrunch disrupt, Turing test, Watson beat the top human players on Jeopardy!

Facebook snapped up LeCun to lead its AI efforts. Bengio remained an independent academic and founded the world’s largest academic research institute for deep learning—the Montreal Institute for Learning Algorithms (MILA). LeCun was gratified to see that the experts who had once shunned his methods had come around. “They said, ‘Okay, now we buy it,’” LeCun later told a reporter. “‘That’s it, now—you won.’” Image recognition was the first problem to succumb to deep learning’s powers. But with the efficacy of the technique no longer in doubt, many of its practitioners turned to an even more enticing task than identifying pictures: understanding words.

…

They might not correspond to meanings on human-understandable terms; they were the dimensions that proved most useful to the neural network when it sifted through the data. The beauty of deep learning is that—whether with images, the sounds of speech, or the meanings of words—humans don’t have to pick out the key identifying features. That task inevitably eludes our grasp, says Steve Young, a senior member of the technical staff for Siri and a professor of information engineering at Cambridge University. “Deep learning,” he says, “just avoids the problem by essentially throwing the entire signal into the classifier and letting the classifier work out what features are significant.”

…

Today Siri pulls from an archive of more than a million sound samples, many of which are as small as half a phoneme. The system uses deep learning to select the optimal sound units—anywhere from a dozen to a hundred or more per sentence—so the puzzle pieces fit cleanly. Because it is trained on examples of real people speaking, the neural network also expresses prosody. Alex Acero, who leads the Siri speech team, says that his ultimate aspiration is to make the assistant sound as natural as the Scarlett Johansson–voiced one in the movie Her. The emergence of powerful deep learning techniques for synthesizing voices means, among other things, that these voices are proliferating.

The Coming Wave: Technology, Power, and the Twenty-First Century's Greatest Dilemma by Mustafa Suleyman

"World Economic Forum" Davos, 23andMe, 3D printing, active measures, Ada Lovelace, additive manufacturing, agricultural Revolution, AI winter, air gap, Airbnb, Alan Greenspan, algorithmic bias, Alignment Problem, AlphaGo, Alvin Toffler, Amazon Web Services, Anthropocene, artificial general intelligence, Asilomar, Asilomar Conference on Recombinant DNA, ASML, autonomous vehicles, backpropagation, barriers to entry, basic income, benefit corporation, Big Tech, biodiversity loss, bioinformatics, Bletchley Park, Blitzscaling, Boston Dynamics, business process, business process outsourcing, call centre, Capital in the Twenty-First Century by Thomas Piketty, ChatGPT, choice architecture, circular economy, classic study, clean tech, cloud computing, commoditize, computer vision, coronavirus, corporate governance, correlation does not imply causation, COVID-19, creative destruction, CRISPR, critical race theory, crowdsourcing, cryptocurrency, cuban missile crisis, data science, decarbonisation, deep learning, deepfake, DeepMind, deindustrialization, dematerialisation, Demis Hassabis, disinformation, drone strike, drop ship, dual-use technology, Easter island, Edward Snowden, effective altruism, energy transition, epigenetics, Erik Brynjolfsson, Ernest Rutherford, Extinction Rebellion, facts on the ground, failed state, Fairchild Semiconductor, fear of failure, flying shuttle, Ford Model T, future of work, general purpose technology, Geoffrey Hinton, global pandemic, GPT-3, GPT-4, hallucination problem, hive mind, hype cycle, Intergovernmental Panel on Climate Change (IPCC), Internet Archive, Internet of things, invention of the wheel, job automation, John Maynard Keynes: technological unemployment, John von Neumann, Joi Ito, Joseph Schumpeter, Kickstarter, lab leak, large language model, Law of Accelerating Returns, Lewis Mumford, license plate recognition, lockdown, machine readable, Marc Andreessen, meta-analysis, microcredit, move 37, Mustafa Suleyman, mutually assured destruction, new economy, Nick Bostrom, Nikolai Kondratiev, off grid, OpenAI, paperclip maximiser, personalized medicine, Peter Thiel, planetary scale, plutocrats, precautionary principle, profit motive, prompt engineering, QAnon, quantum entanglement, ransomware, Ray Kurzweil, Recombinant DNA, Richard Feynman, Robert Gordon, Ronald Reagan, Sam Altman, Sand Hill Road, satellite internet, Silicon Valley, smart cities, South China Sea, space junk, SpaceX Starlink, stealth mode startup, stem cell, Stephen Fry, Steven Levy, strong AI, synthetic biology, tacit knowledge, tail risk, techlash, techno-determinism, technoutopianism, Ted Kaczynski, the long tail, The Rise and Fall of American Growth, Thomas Malthus, TikTok, TSMC, Turing test, Tyler Cowen, Tyler Cowen: Great Stagnation, universal basic income, uranium enrichment, warehouse robotics, William MacAskill, working-age population, world market for maybe five computers, zero day

The breakthrough moment took nearly half a century, finally arriving in 2012 in the form of a system called AlexNet. AlexNet was powered by the resurgence of an old technique that has now become fundamental to AI, one that has supercharged the field and was integral to us at DeepMind: deep learning. Deep learning uses neural networks loosely modeled on those of the human brain. In simple terms, these systems “learn” when their networks are “trained” on large amounts of data. In the case of AlexNet, the training data consisted of images. Each red, green, or blue pixel is given a value, and the resulting array of numbers is fed into the network as an input.

…

In 1987 there were just ninety academic papers published at Neural Information Processing Systems, at what became the field’s leading conference. By the 2020s there were almost two thousand. In the last six years there was a six-fold increase in the number of papers published on deep learning alone, tenfold if you widen the view to machine learning as a whole. With the blossoming of deep learning, billions of dollars poured into AI research at academic institutions and private and public companies. Starting in the 2010s, the buzz, indeed the hype, around AI was back, stronger than ever, making headlines and pushing the frontiers of what’s possible.

…

GO TO NOTE REFERENCE IN TEXT In 2012, AlexNet beat Jerry Wei, “AlexNet: The Architecture That Challenged CNNs,” Towards Data Science, July 2, 2019, towardsdatascience.com/​alexnet-the-architecture-that-challenged-cnns-e406d5297951. GO TO NOTE REFERENCE IN TEXT Thanks to deep learning Chanan Bos, “Tesla’s New HW3 Self-Driving Computer—It’s a Beast,” CleanTechnica, June 15, 2019, cleantechnica.com/​2019/​06/​15/​teslas-new-hw3-self-driving-computer-its-a-beast-cleantechnica-deep-dive. GO TO NOTE REFERENCE IN TEXT It helps fly drones Jeffrey De Fauw et al., “Clinically Applicable Deep Learning for Diagnosis and Referral in Retinal Disease,” Nature Medicine, Aug. 13, 2018, www.nature.com/​articles/​s41591-018-0107-6. GO TO NOTE REFERENCE IN TEXT By the 2020s there were almost two thousand “Advances in Neural Information Processing Systems,” NeurIPS, papers.nips.cc.

Rationality: What It Is, Why It Seems Scarce, Why It Matters by Steven Pinker

affirmative action, Albert Einstein, autonomous vehicles, availability heuristic, Ayatollah Khomeini, backpropagation, basic income, behavioural economics, belling the cat, Black Lives Matter, butterfly effect, carbon tax, Cass Sunstein, choice architecture, classic study, clean water, Comet Ping Pong, coronavirus, correlation coefficient, correlation does not imply causation, COVID-19, critical race theory, crowdsourcing, cuban missile crisis, Daniel Kahneman / Amos Tversky, data science, David Attenborough, deep learning, defund the police, delayed gratification, disinformation, Donald Trump, Dr. Strangelove, Easter island, effective altruism, en.wikipedia.org, Erdős number, Estimating the Reproducibility of Psychological Science, fake news, feminist movement, framing effect, George Akerlof, George Floyd, germ theory of disease, high batting average, if you see hoof prints, think horses—not zebras, index card, Jeff Bezos, job automation, John Nash: game theory, John von Neumann, libertarian paternalism, Linda problem, longitudinal study, loss aversion, Mahatma Gandhi, meta-analysis, microaggression, Monty Hall problem, Nash equilibrium, New Journalism, Paul Erdős, Paul Samuelson, Peter Singer: altruism, Pierre-Simon Laplace, placebo effect, post-truth, power law, QAnon, QWERTY keyboard, Ralph Waldo Emerson, randomized controlled trial, replication crisis, Richard Thaler, scientific worldview, selection bias, social discount rate, social distancing, Social Justice Warrior, Stanford marshmallow experiment, Steve Bannon, Steven Pinker, sunk-cost fallacy, TED Talk, the scientific method, Thomas Bayes, Tragedy of the Commons, trolley problem, twin studies, universal basic income, Upton Sinclair, urban planning, Walter Mischel, yellow journalism, zero-sum game

Computer scientists could put multilayer networks on megavitamins, giving them two, fifteen, even a thousand hidden layers, and training them on billions or even trillions of examples. The networks are called deep learning systems because of the number of layers between the input and the output (they’re not deep in the sense of understanding anything). These networks are powering “the great AI awakening” we are living through, which is giving us the first serviceable products for speech and image recognition, question-answering, translation, and other humanlike feats.33 Deep learning networks often outperform GOFAI (good old-fashioned artificial intelligence), which executes logic-like deductions on hand-coded propositions and rules.34 The contrast in the way they work is stark: unlike logical inference, the inner workings of a neural network are inscrutable.

…

That is why many technology critics fear that as AI systems are entrusted with decisions about the fates of people, they could perpetuate biases that no one can identify and uproot.35 In 2018 Henry Kissinger warned that since deep learning systems don’t work on propositions we can examine and justify, they portend the end of the Enlightenment.36 That is a stretch, but the contrast between logic and neural computation is clear. Is the human brain a big deep learning network? Certainly not, for many reasons, but the similarities are illuminating. The brain has around a hundred billion neurons connected by a hundred trillion synapses, and by the time we are eighteen we have been absorbing examples from our environments for more than three hundred million waking seconds.

…

New York: Basic Books. Marcus, G. F. 2000. Two kinds of representation. In E. Dietrich & A. B. Markman, eds., Cognitive dynamics: Conceptual and representational change in humans and machines. Mahwah, NJ: Erlbaum. Marcus, G. F. 2018. The deepest problem with deep learning. Medium, Dec. 1. https://medium.com/@GaryMarcus/the-deepest-problem-with-deep-learning-91c5991f5695. Marcus, G. F., & Davis, E. 2019. Rebooting AI: Building artificial intelligence we can trust. New York: Pantheon. Marlowe, F. 2010. The Hadza: Hunter-gatherers of Tanzania. Berkeley: University of California Press. Martin, G.

Machine Learning Design Patterns: Solutions to Common Challenges in Data Preparation, Model Building, and MLOps by Valliappa Lakshmanan, Sara Robinson, Michael Munn

A Pattern Language, Airbnb, algorithmic trading, automated trading system, business intelligence, business logic, business process, combinatorial explosion, computer vision, continuous integration, COVID-19, data science, deep learning, DevOps, discrete time, en.wikipedia.org, Hacker News, industrial research laboratory, iterative process, Kubernetes, machine translation, microservices, mobile money, natural language processing, Netflix Prize, optical character recognition, pattern recognition, performance metric, recommendation engine, ride hailing / ride sharing, selection bias, self-driving car, sentiment analysis, speech recognition, statistical model, the payments system, web application

Neural networks with more than one hidden layer (layers other than the input and output layer) are classified as deep learning (see Figure 1-1). Machine learning models, regardless of how they are depicted visually, are mathematical functions and can therefore be implemented from scratch using a numerical software package. However, ML engineers in industry tend to employ one of several open source frameworks designed to provide intuitive APIs for building models. The majority of our examples will use TensorFlow, an open source machine learning framework created by Google with a focus on deep learning models. Within the TensorFlow library, we’ll be using the Keras API in our examples, which can be imported through tensorflow.keras.

…

One bit of intuition as to why this works comes from the Uniform Approximation Theorem of deep learning, which, loosely put, states that any function (and its derivatives) can be approximated by a neural network with at least one hidden layer and any “squashing” activation function, like sigmoid. This means that no matter what function we are given, so long as it’s relatively well behaved, there exists a neural network with just one hidden layer that approximates that function as closely as we want.1 Deep learning approaches to solving differential equations or complex dynamical systems aim to represent a function defined implicitly by a differential equation, or system of equations, using a neural network.

…

While increasing the resolution requires substantially more compute power using finite-difference methods, the neural network is able to maintain high performance with only marginal additional cost. Techniques like the Deep Galerkin Method can then use deep learning to provide a mesh-free approximation of the solution to the given PDE. In this way, solving the PDE is reduced to a chained optimization problem (see “Design Pattern 8: Cascade ”). Deep Galerkin Method The Deep Galerkin Method is a deep learning algorithm for solving partial differential equations. The algorithm is similar in spirit to Galerkin methods used in the field of numeric analysis, where the solution is approximated using a neural network instead of a linear combination of basis functions.

Exponential Organizations: Why New Organizations Are Ten Times Better, Faster, and Cheaper Than Yours (And What to Do About It) by Salim Ismail, Yuri van Geest

23andMe, 3D printing, Airbnb, Amazon Mechanical Turk, Amazon Web Services, anti-fragile, augmented reality, autonomous vehicles, Baxter: Rethink Robotics, behavioural economics, Ben Horowitz, bike sharing, bioinformatics, bitcoin, Black Swan, blockchain, Blue Ocean Strategy, book value, Burning Man, business intelligence, business process, call centre, chief data officer, Chris Wanstrath, circular economy, Clayton Christensen, clean water, cloud computing, cognitive bias, collaborative consumption, collaborative economy, commoditize, corporate social responsibility, cross-subsidies, crowdsourcing, cryptocurrency, dark matter, data science, Dean Kamen, deep learning, DeepMind, dematerialisation, discounted cash flows, disruptive innovation, distributed ledger, driverless car, Edward Snowden, Elon Musk, en.wikipedia.org, Ethereum, ethereum blockchain, fail fast, game design, gamification, Google Glasses, Google Hangouts, Google X / Alphabet X, gravity well, hiring and firing, holacracy, Hyperloop, industrial robot, Innovator's Dilemma, intangible asset, Internet of things, Iridium satellite, Isaac Newton, Jeff Bezos, Joi Ito, Kevin Kelly, Kickstarter, knowledge worker, Kodak vs Instagram, Law of Accelerating Returns, Lean Startup, life extension, lifelogging, loose coupling, loss aversion, low earth orbit, Lyft, Marc Andreessen, Mark Zuckerberg, market design, Max Levchin, means of production, Michael Milken, minimum viable product, natural language processing, Netflix Prize, NetJets, Network effects, new economy, Oculus Rift, offshore financial centre, PageRank, pattern recognition, Paul Graham, paypal mafia, peer-to-peer, peer-to-peer model, Peter H. Diamandis: Planetary Resources, Peter Thiel, Planet Labs, prediction markets, profit motive, publish or perish, radical decentralization, Ray Kurzweil, recommendation engine, RFID, ride hailing / ride sharing, risk tolerance, Ronald Coase, Rutger Bregman, Salesforce, Second Machine Age, self-driving car, sharing economy, Silicon Valley, skunkworks, Skype, smart contracts, Snapchat, social software, software is eating the world, SpaceShipOne, speech recognition, stealth mode startup, Stephen Hawking, Steve Jobs, Steve Jurvetson, subscription business, supply-chain management, synthetic biology, TaskRabbit, TED Talk, telepresence, telepresence robot, the long tail, Tony Hsieh, transaction costs, Travis Kalanick, Tyler Cowen, Tyler Cowen: Great Stagnation, uber lyft, urban planning, Virgin Galactic, WikiLeaks, winner-take-all economy, X Prize, Y Combinator, zero-sum game

After allowing it to browse ten million randomly selected YouTube video thumbnails for three days, the network began to recognize cats, without actually knowing the concept of “cats.” Importantly, this was without any human intervention or input. In the two years since, the capabilities of Deep Learning have improved considerably. Today, in addition to improving speech recognition, creating a more effective search engine (Ray Kurzweil is working on this within Google) and identifying individual objects, Deep Learning algorithms can also detect particular episodes in videos and even describe them in text, all without human input. Deep Learning algorithms can even play video games by figuring out the rules of the game and then optimizing performance. Think about the implications of this revolutionary breakthrough.

…

The contest ended early, in September 2009, when one of the 44,014 valid submissions achieved the goal and was awarded the prize. Deep Learning is a new and exciting subset of Machine Learning based on neural net technology. It allows a machine to discover new patterns without being exposed to any historical or training data. Leading startups in this space are DeepMind, bought by Google in early 2014 for $500 million, back when DeepMind had just thirteen employees, and Vicarious, funded with investment from Elon Musk, Jeff Bezos and Mark Zuckerberg. Twitter, Baidu, Microsoft and Facebook are also heavily invested in this area. Deep Learning algorithms rely on discovery and self-indexing, and operate in much the same way that a baby learns first sounds, then words, then sentences and even languages.

…

To implement platforms, ExOs follow four steps in terms of data and APIs: Gather: The algorithmic process starts with harnessing data, which is gathered via sensors, people, or imported from public datasets. Organize: The next step is to organize the data. This is known as ETL (extract, transform and load). Apply: Once the data is accessible, algorithms such as machine or deep learning extract insights, identify trends and tune new algorithms. These are realized via tools such as Hadoop and Pivotal, or even (open source) deep learning algorithms like DeepMind or Skymind. Expose: The final step is exposing the data in the form of an open platform. Open data and APIs can be used such that an ExO’s community develops valuable services, new functionalities and innovations layered on top of the platform by remixing published data with their own.

System Error by Rob Reich

"Friedman doctrine" OR "shareholder theory", "World Economic Forum" Davos, 2021 United States Capitol attack, A Declaration of the Independence of Cyberspace, Aaron Swartz, AI winter, Airbnb, airport security, Alan Greenspan, Albert Einstein, algorithmic bias, AlphaGo, AltaVista, artificial general intelligence, Automated Insights, autonomous vehicles, basic income, Ben Horowitz, Berlin Wall, Bernie Madoff, Big Tech, bitcoin, Blitzscaling, Cambridge Analytica, Cass Sunstein, clean water, cloud computing, computer vision, contact tracing, contact tracing app, coronavirus, corporate governance, COVID-19, creative destruction, CRISPR, crowdsourcing, data is the new oil, data science, decentralized internet, deep learning, deepfake, DeepMind, deplatforming, digital rights, disinformation, disruptive innovation, Donald Knuth, Donald Trump, driverless car, dual-use technology, Edward Snowden, Elon Musk, en.wikipedia.org, end-to-end encryption, Fairchild Semiconductor, fake news, Fall of the Berlin Wall, Filter Bubble, financial engineering, financial innovation, fulfillment center, future of work, gentrification, Geoffrey Hinton, George Floyd, gig economy, Goodhart's law, GPT-3, Hacker News, hockey-stick growth, income inequality, independent contractor, informal economy, information security, Jaron Lanier, Jeff Bezos, Jim Simons, jimmy wales, job automation, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John Perry Barlow, Lean Startup, linear programming, Lyft, Marc Andreessen, Mark Zuckerberg, meta-analysis, minimum wage unemployment, Monkeys Reject Unequal Pay, move fast and break things, Myron Scholes, Network effects, Nick Bostrom, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, NP-complete, Oculus Rift, OpenAI, Panopticon Jeremy Bentham, Parler "social media", pattern recognition, personalized medicine, Peter Thiel, Philippa Foot, premature optimization, profit motive, quantitative hedge fund, race to the bottom, randomized controlled trial, recommendation engine, Renaissance Technologies, Richard Thaler, ride hailing / ride sharing, Ronald Reagan, Sam Altman, Sand Hill Road, scientific management, self-driving car, shareholder value, Sheryl Sandberg, Shoshana Zuboff, side project, Silicon Valley, Snapchat, social distancing, Social Responsibility of Business Is to Increase Its Profits, software is eating the world, spectrum auction, speech recognition, stem cell, Steve Jobs, Steven Levy, strong AI, superintelligent machines, surveillance capitalism, Susan Wojcicki, tech billionaire, tech worker, techlash, technoutopianism, Telecommunications Act of 1996, telemarketer, The Future of Employment, TikTok, Tim Cook: Apple, traveling salesman, Triangle Shirtwaist Factory, trolley problem, Turing test, two-sided market, Uber and Lyft, uber lyft, ultimatum game, union organizing, universal basic income, washing machines reduced drudgery, Watson beat the top human players on Jeopardy!, When a measure becomes a target, winner-take-all economy, Y Combinator, you are the product

The extent to which AI technology will threaten to displace highly skilled workers such as doctors ultimately remains to be seen. A 2019 study in the United Kingdom summarized the state of affairs, stating “Our review found the diagnostic performance of deep learning models to be equivalent to that of health-care professionals,” but then went on to conclude that “poor reporting is prevalent in deep learning studies, which limits reliable interpretation of the reported diagnostic accuracy.” In other words, the deep learning models may be able to match human performance at the narrow task of making a diagnosis from an X-ray, but the fact that such studies don’t then use the results of the algorithm in an actual medical setting means it’s not possible to determine whether the model’s prediction would have actually led to a better outcome for the patient.

…

., “Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning,” CheXNet, December 25, 2017, http://arxiv.org/abs/1711.05225. “people should stop training radiologists”: Geoff Hinton, “Geoff Hinton: On Radiology,” Creative Destruction Lab, uploaded to YouTube November 24, 2016, https://www.youtube.com/watch?v=2HMPRXstSvQ. the work radiologists and other medical professionals do: Hugh Harvey, “Why AI Will Not Replace Radiologists,” Medium, April 7, 2018, https://towardsdatascience.com/why-ai-will-not-replace-radiologists-c7736f2c7d80. “deep learning models to be equivalent”: Xiaoxuan Liu et al., “A Comparison of Deep Learning Performance Against Health-Care Professionals in Detecting Diseases from Medical Imaging: A Systematic Review and Meta-Analysis,” Lancet Digital Health 1, no. 6 (October 1, 2019): e271–97, https://doi.org/10.1016/S2589-7500(19)30123-2.

…

AI systems can identify patterns in huge pools of data that humans can’t discern and can therefore frequently make more accurate predictions. But such systems are often black boxes, unable to explain why they generate particular outputs. And the scientists who build the systems can’t always explain the outputs either, making the decisions inscrutable. Deep learning refers not to the capacity to generate insights but to a spatial metaphor for the architecture of the AI system. The idea behind deep learning is that inputs to a system form sets of simple patterns that are then combined into increasingly more complex patterns using patterns derived from the previous layer. The moniker deep comes from the fact that such systems now contain many more layers than they did just a decade ago—a result of greater computational power to model the patterns in each layer and vast increases in data that enable these more complex patterns to be discovered.

A Thousand Brains: A New Theory of Intelligence by Jeff Hawkins

AI winter, Albert Einstein, artificial general intelligence, carbon-based life, clean water, cloud computing, deep learning, different worldview, discovery of DNA, Doomsday Clock, double helix, en.wikipedia.org, estate planning, Geoffrey Hinton, Jeff Hawkins, PalmPilot, Search for Extraterrestrial Intelligence, self-driving car, sensor fusion, Silicon Valley, superintelligent machines, the scientific method, Thomas Kuhn: the structure of scientific revolutions, Turing machine, Turing test

The current wave of AI has attracted thousands of researchers and billions of dollars of investment. Almost all these people and dollars are being applied to improving deep learning technologies. Will this investment lead to human-level machine intelligence, or are deep learning technologies fundamentally limited, leading us once again to reinvent the field of AI? When you are in the middle of a bubble, it is easy to get swept up in the enthusiasm and believe it will go on forever. History suggests we should be cautious. I don’t know how long the current wave of AI will continue to grow. But I do know that deep learning does not put us on the path to creating truly intelligent machines. We can’t get to artificial general intelligence by doing more of what we are currently doing.

…

They claimed that we could not make truly intelligent machines until we solved how to represent everyday knowledge in a computer. Today’s deep learning networks don’t possess knowledge. A Go-playing computer does not know that Go is a game. It doesn’t know the history of the game. It doesn’t know if it is playing against a computer or a human, or what “computer” and “human” mean. Similarly, a deep learning network that labels images may look at an image and say it is a cat. But the computer has limited knowledge of cats. It doesn’t know that cats are animals, or that they have tails, legs, and lungs. It doesn’t know about cat people versus dog people, or that cats purr and shed fur. All the deep learning network does is determine that a new image is similar to previously seen images that were labeled “cat.”

…

AI scientists disagree as to whether these language networks possess true knowledge or are just mimicking humans by remembering the statistics of millions of words. I don’t believe any kind of deep learning network will achieve the goal of AGI if the network doesn’t model the world the way a brain does. Deep learning networks work well, but not because they solved the knowledge representation problem. They work well because they avoided it completely, relying on statistics and lots of data instead. How deep learning networks work is clever, their performance is impressive, and they are commercially valuable. I am only pointing out that they don’t possess knowledge and, therefore, are not on the path to having the ability of a five-year-old child.

Hands-On Machine Learning With Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems by Aurélien Géron

AlphaGo, Amazon Mechanical Turk, Anton Chekhov, backpropagation, combinatorial explosion, computer vision, constrained optimization, correlation coefficient, crowdsourcing, data science, deep learning, DeepMind, don't repeat yourself, duck typing, Elon Musk, en.wikipedia.org, friendly AI, Geoffrey Hinton, ImageNet competition, information retrieval, iterative process, John von Neumann, Kickstarter, machine translation, natural language processing, Netflix Prize, NP-complete, OpenAI, optical character recognition, P = NP, p-value, pattern recognition, pull request, recommendation engine, self-driving car, sentiment analysis, SpamAssassin, speech recognition, stochastic process

If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 978-1-491-96229-9 [LSI] Preface The Machine Learning Tsunami In 2006, Geoffrey Hinton et al. published a paper1 showing how to train a deep neural network capable of recognizing handwritten digits with state-of-the-art precision (>98%). They branded this technique “Deep Learning.” Training a deep neural net was widely considered impossible at the time,2 and most researchers had abandoned the idea since the 1990s. This paper revived the interest of the scientific community and before long many new papers demonstrated that Deep Learning was not only possible, but capable of mind-blowing achievements that no other Machine Learning (ML) technique could hope to match (with the help of tremendous computing power and great amounts of data).

…

Part II, Neural Networks and Deep Learning, covers the following topics: What are neural nets? What are they good for? Building and training neural nets using TensorFlow. The most important neural net architectures: feedforward neural nets, convolutional nets, recurrent nets, long short-term memory (LSTM) nets, and autoencoders. Techniques for training deep neural nets. Scaling neural networks for huge datasets. Reinforcement learning. The first part is based mostly on Scikit-Learn while the second part uses TensorFlow. Caution Don’t jump into deep waters too hastily: while Deep Learning is no doubt one of the most exciting areas in Machine Learning, you should master the fundamentals first.

…

Moreover, most problems can be solved quite well using simpler techniques such as Random Forests and Ensemble methods (discussed in Part I). Deep Learning is best suited for complex problems such as image recognition, speech recognition, or natural language processing, provided you have enough data, computing power, and patience. Other Resources Many resources are available to learn about Machine Learning. Andrew Ng’s ML course on Coursera and Geoffrey Hinton’s course on neural networks and Deep Learning are amazing, although they both require a significant time investment (think months). There are also many interesting websites about Machine Learning, including of course Scikit-Learn’s exceptional User Guide.

What to Think About Machines That Think: Today's Leading Thinkers on the Age of Machine Intelligence by John Brockman

Adam Curtis, agricultural Revolution, AI winter, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, algorithmic trading, Anthropocene, artificial general intelligence, augmented reality, autism spectrum disorder, autonomous vehicles, backpropagation, basic income, behavioural economics, bitcoin, blockchain, bread and circuses, Charles Babbage, clean water, cognitive dissonance, Colonization of Mars, complexity theory, computer age, computer vision, constrained optimization, corporate personhood, cosmological principle, cryptocurrency, cuban missile crisis, Danny Hillis, dark matter, data science, deep learning, DeepMind, Demis Hassabis, digital capitalism, digital divide, digital rights, discrete time, Douglas Engelbart, driverless car, Elon Musk, Emanuel Derman, endowment effect, epigenetics, Ernest Rutherford, experimental economics, financial engineering, Flash crash, friendly AI, functional fixedness, global pandemic, Google Glasses, Great Leap Forward, Hans Moravec, hive mind, Ian Bogost, income inequality, information trail, Internet of things, invention of writing, iterative process, James Webb Space Telescope, Jaron Lanier, job automation, Johannes Kepler, John Markoff, John von Neumann, Kevin Kelly, knowledge worker, Large Hadron Collider, lolcat, loose coupling, machine translation, microbiome, mirror neurons, Moneyball by Michael Lewis explains big data, Mustafa Suleyman, natural language processing, Network effects, Nick Bostrom, Norbert Wiener, paperclip maximiser, pattern recognition, Peter Singer: altruism, phenotype, planetary scale, Ray Kurzweil, Recombinant DNA, recommendation engine, Republic of Letters, RFID, Richard Thaler, Rory Sutherland, Satyajit Das, Search for Extraterrestrial Intelligence, self-driving car, sharing economy, Silicon Valley, Skype, smart contracts, social intelligence, speech recognition, statistical model, stem cell, Stephen Hawking, Steve Jobs, Steven Pinker, Stewart Brand, strong AI, Stuxnet, superintelligent machines, supervolcano, synthetic biology, systems thinking, tacit knowledge, TED Talk, the scientific method, The Wisdom of Crowds, theory of mind, Thorstein Veblen, too big to fail, Turing machine, Turing test, Von Neumann architecture, Watson beat the top human players on Jeopardy!, We are as Gods, Y2K

Some of those patterns are complex, but most are fairly simple. Great effort goes into parsing our speech and deciphering our handwriting. The current fad in thinking machines goes by the name of deep learning. When I first heard of deep learning, I was excited by the idea that machines were finally going to reveal to us deep aspects of existence—truth, beauty, and love. I was rapidly disabused. The deep in deep learning refers to the architecture of the machines doing the learning: They consist of many layers of interlocking logical elements, analogous to the “deep” layers of interlocking neurons in the brain.

…

Today’s algorithm has nothing like human-level competence in understanding images. Work is under way to add focus of attention and handling of consistent spatial structure to deep learning. That’s the hard work of science and research, and we have no idea how hard it will be, nor how long it will take, nor whether the whole approach will reach a dead end. It took some thirty years to go from backpropagation to deep learning, but along the way many researchers were sure there was no future in backpropagation. They were wrong, but it wouldn’t have been surprising if they were right, as we knew all along that the backpropagation algorithm is not what happens inside people’s heads.

…

After thirty years of research, a million-times improvement in computer power, and vast data sets from the Internet, we now know the answer to this question: Neural networks scaled up to twelve layers deep, with billions of connections, are outperforming the best algorithms in computer vision for object recognition and have revolutionized speech recognition. It’s rare for any algorithm to scale this well, which suggests that they may soon be able to solve even more difficult problems. Recent breakthroughs have been made that allow the application of deep learning to natural-language processing. Deep recurrent networks with short-term memory were trained to translate English sentences into French sentences at high levels of performance. Other deep-learning networks could create English captions for the content of images with surprising and sometimes amusing acumen. Supervised learning using deep networks is a step forward, but still far from achieving general intelligence.

Leadership by Algorithm: Who Leads and Who Follows in the AI Era? by David de Cremer

"Friedman doctrine" OR "shareholder theory", algorithmic bias, algorithmic management, AlphaGo, bitcoin, blockchain, business climate, business process, Computing Machinery and Intelligence, corporate governance, data is not the new oil, data science, deep learning, DeepMind, Donald Trump, Elon Musk, fake news, future of work, job automation, Kevin Kelly, Mark Zuckerberg, meta-analysis, Norbert Wiener, pattern recognition, Peter Thiel, race to the bottom, robotic process automation, Salesforce, scientific management, shareholder value, Silicon Valley, Social Responsibility of Business Is to Increase Its Profits, Stephen Hawking, The Future of Employment, Turing test, work culture , workplace surveillance , zero-sum game

AI witnessed a comeback in the last decade, primarily because the world woke up to the realization that deep learning by machines is possible to the level where they can actually perform many tasks better than humans. Where did this wake-up call come from? From a simple game called Go. In 2016, AlphaGo, a program developed by Google DeepMind, beat the human world champion in the Chinese board game, Go. This was a surprise to many, as Go – because of its complexity – was considered the territory of human, not AI, victors. In a decade where our human desire to connect globally, execute tasks faster, and accumulate massive amounts of data, was omnipresent, such deep learning capabilities were, of course, quickly embraced.

…

‘Most of AI’s Business Uses Will Be in Two Areas.’ Harvard Business Review. July 20. Retrieved from: https://hbr.org/2018/07/most-of-ais-business-uses-will-be-in-two-areas 8 McKinsey (2018). ‘Notes from the AI frontier: Applications and value of deep learning.’ Retrieved from: https://www.mckinsey.com/featured-insights/artificial-intelligence/notes-from-the-ai-frontier-applications-and-value-of-deep-learning 9 Bloomberg (2018, January 15th). ‘Alibaba's AI Outguns Humans in Reading Test.’ Retrieved from https://www.bloomberg.com/news/articles/2018-01-15/alibaba-s-ai-outgunned-humans-in-key-stanford-reading-test 10 Gee, K. (2017).

…

In that respect, an interesting study from the US National Bureau of Economic Research demonstrated that low-skill service-sector workers (where retention rates are low) stayed in the job 15% longer when an algorithm was used to judge their employability.¹⁴ Automation and innovation Automation and the corresponding use of algorithms with deep learning abilities are also penetrating other industries. The legal sector is another area where many discussions are taking place about how and whether to automate services. Legal counsellors have started to use automated advisors to contest relatively small fines such as parking tickets. The legal sector is also considering the use of AI to help judges go through evidence collected to reach a verdict in court cases.

The Book of Why: The New Science of Cause and Effect by Judea Pearl, Dana Mackenzie

affirmative action, Albert Einstein, AlphaGo, Asilomar, Bayesian statistics, computer age, computer vision, Computing Machinery and Intelligence, confounding variable, correlation coefficient, correlation does not imply causation, Daniel Kahneman / Amos Tversky, data science, deep learning, DeepMind, driverless car, Edmond Halley, Elon Musk, en.wikipedia.org, experimental subject, Great Leap Forward, Gregor Mendel, Isaac Newton, iterative process, John Snow's cholera map, Loebner Prize, loose coupling, Louis Pasteur, Menlo Park, Monty Hall problem, pattern recognition, Paul Erdős, personalized medicine, Pierre-Simon Laplace, placebo effect, Plato's cave, prisoner's dilemma, probability theory / Blaise Pascal / Pierre de Fermat, randomized controlled trial, Recombinant DNA, selection bias, self-driving car, seminal paper, Silicon Valley, speech recognition, statistical model, Stephen Hawking, Steve Jobs, strong AI, The Design of Experiments, the scientific method, Thomas Bayes, Turing test

Some readers may be surprised to see that I have placed present-day learning machines squarely on rung one of the Ladder of Causation, sharing the wisdom of an owl. We hear almost every day, it seems, about rapid advances in machine learning systems—self-driving cars, speech-recognition systems, and, especially in recent years, deep-learning algorithms (or deep neural networks). How could they still be only at level one? The successes of deep learning have been truly remarkable and have caught many of us by surprise. Nevertheless, deep learning has succeeded primarily by showing that certain questions or tasks we thought were difficult are in fact not. It has not addressed the truly difficult questions that continue to prevent us from achieving humanlike AI.

…

Another advantage causal models have that data mining and deep learning lack is adaptability. Note that in Figure I.1, the estimand is computed on the basis of the causal model alone, prior to an examination of the specifics of the data. This makes the causal inference engine supremely adaptable, because the estimand computed is good for any data that are compatible with the qualitative model, regardless of the numerical relationships among the variables. To see why this adaptability is important, compare this engine with a learning agent—in this instance a human, but in other cases perhaps a deep-learning algorithm or maybe a human using a deep-learning algorithm—trying to learn solely from the data.

…

A few months later it played sixty online games against top human players without losing a single one, and in 2017 it was officially retired after beating the current world champion, Ke Jie. The one game it lost to Sedol is the only one it will ever lose to a human. All of this is exciting, and the results leave no doubt: deep learning works for certain tasks. But it is the antithesis of transparency. Even AlphaGo’s programmers cannot tell you why the program plays so well. They knew from experience that deep networks have been successful at tasks in computer vision and speech recognition. Nevertheless, our understanding of deep learning is completely empirical and comes with no guarantees. The AlphaGo team could not have predicted at the outset that the program would beat the best human in a year, or two, or five.

Machine, Platform, Crowd: Harnessing Our Digital Future by Andrew McAfee, Erik Brynjolfsson

"World Economic Forum" Davos, 3D printing, additive manufacturing, AI winter, Airbnb, airline deregulation, airport security, Albert Einstein, algorithmic bias, AlphaGo, Amazon Mechanical Turk, Amazon Web Services, Andy Rubin, AOL-Time Warner, artificial general intelligence, asset light, augmented reality, autism spectrum disorder, autonomous vehicles, backpropagation, backtesting, barriers to entry, behavioural economics, bitcoin, blockchain, blood diamond, British Empire, business cycle, business process, carbon footprint, Cass Sunstein, centralized clearinghouse, Chris Urmson, cloud computing, cognitive bias, commoditize, complexity theory, computer age, creative destruction, CRISPR, crony capitalism, crowdsourcing, cryptocurrency, Daniel Kahneman / Amos Tversky, data science, Dean Kamen, deep learning, DeepMind, Demis Hassabis, discovery of DNA, disintermediation, disruptive innovation, distributed ledger, double helix, driverless car, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Ethereum, ethereum blockchain, everywhere but in the productivity statistics, Evgeny Morozov, fake news, family office, fiat currency, financial innovation, general purpose technology, Geoffrey Hinton, George Akerlof, global supply chain, Great Leap Forward, Gregor Mendel, Hernando de Soto, hive mind, independent contractor, information asymmetry, Internet of things, inventory management, iterative process, Jean Tirole, Jeff Bezos, Jim Simons, jimmy wales, John Markoff, joint-stock company, Joseph Schumpeter, Kickstarter, Kiva Systems, law of one price, longitudinal study, low interest rates, Lyft, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, Marc Andreessen, Marc Benioff, Mark Zuckerberg, meta-analysis, Mitch Kapor, moral hazard, multi-sided market, Mustafa Suleyman, Myron Scholes, natural language processing, Network effects, new economy, Norbert Wiener, Oculus Rift, PageRank, pattern recognition, peer-to-peer lending, performance metric, plutocrats, precision agriculture, prediction markets, pre–internet, price stability, principal–agent problem, Project Xanadu, radical decentralization, Ray Kurzweil, Renaissance Technologies, Richard Stallman, ride hailing / ride sharing, risk tolerance, Robert Solow, Ronald Coase, Salesforce, Satoshi Nakamoto, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Skype, slashdot, smart contracts, Snapchat, speech recognition, statistical model, Steve Ballmer, Steve Jobs, Steven Pinker, supply-chain management, synthetic biology, tacit knowledge, TaskRabbit, Ted Nelson, TED Talk, the Cathedral and the Bazaar, The Market for Lemons, The Nature of the Firm, the strength of weak ties, Thomas Davenport, Thomas L Friedman, too big to fail, transaction costs, transportation-network company, traveling salesman, Travis Kalanick, Two Sigma, two-sided market, Tyler Cowen, Uber and Lyft, Uber for X, uber lyft, ubercab, Vitalik Buterin, warehouse robotics, Watson beat the top human players on Jeopardy!, winner-take-all economy, yield management, zero day

November 11, 2016, http://metricviews.org.uk/2007/11/how-big-hectare. 80 Makoto was impressed: Kaz Sato, “How a Japanese Cucumber Farmer Is Using Deep Learning and TensorFlow,” Google, August 31, 2016, https://cloud.google.com/blog/big-data/2016/08/how-a-japanese-cucumber-farmer-is-using-deep-learning-and-tensorflow. 80 “I can’t wait to try it”: Ibid. 80 “It’s not hyperbole”: Ibid. 80 “If intelligence was a cake”: Carlos E. Perez, “ ‘Predictive Learning’ Is the New Buzzword in Deep Learning,” Intuition Machine, December 6, 2016, https://medium.com/intuitionmachine/predictive-learning-is-the-key-to-deep-learning-acceleration-93e063195fd0#.13qh1nti1. 81 Joshua Brown’s Tesla crashed: Anjali Singhvi and Karl Russell, “Inside the Self-Driving Tesla Fatal Accident,” New York Times, July 12, 2016, https://www.nytimes.com/interactive/2016/07/01/business/inside-tesla-accident.html. 82 it appears that neither Brown: Tesla, “A Tragic Loss,” June 30, 2016, https://www.tesla.com/blog/tragic-loss. 82 “Conventional wisdom would say”: Chris Urmson, “How a Driverless Car Sees the Road,” TED Talk, June 2015, 15:29, https://www.ted.com/talks/chris_urmson_how_a_driverless_car_sees_the_road/transcript?

…

Because both supervised and unsupervised machine learning approaches use the algorithms described by Hinton and his colleagues in their 2006 paper, they’re now commonly called “deep learning” systems. Demonstrations and Deployments Except for a very small number of cases, such as the system LeCun built for recognizing handwritten numbers on checks, the business application of deep learning is only a few years old. But the technique is spreading with extraordinary speed. The software engineer Jeff Dean,** who heads Google’s efforts to use the technology, notes that as recently as 2012 the company was not using it at all to improve products like Search, Gmail, YouTube, or Maps. By the third quarter of 2015, however, deep learning was being used in approximately 1,200 projects across the company, having surpassed the performance of other methods.

…

By 2013, these challenges had been broadly addressed (Erik Bernhardsson, “When Machine Learning Matters,” Erik Bernhardsson [blog], August 5, 2016, https://erikbern.com/2016/08/05/when-machine-learning-matters.html), and the company shifted focus toward using machine learning to deliver highly personalized music recommendations (Jordan Novet, “Spotify Intern Dreams Up Better Music Recommendations through Deep Learning,” VentureBeat, August 6, 2014, http://venturebeat.com/2014/08/06/spotify-intern-dreams-up-better-music-recommendations-through-deep-learning). Spotify launched its algorithm-powered Daily Mix option in September 2016 (Spotify, “Rediscover Your Favorite Music with Daily Mix,” September 27, 2016, https://news.spotify.com/us/2016/09/27/rediscover-your-favorite-music-with-daily-mix).

Data Science from Scratch: First Principles with Python by Joel Grus

backpropagation, confounding variable, correlation does not imply causation, data science, deep learning, Hacker News, higher-order functions, natural language processing, Netflix Prize, p-value, Paul Graham, recommendation engine, SpamAssassin, statistical model

Our documents are our users’ interests, which look like: documents = [ ["Hadoop", "Big Data", "HBase", "Java", "Spark", "Storm", "Cassandra"], ["NoSQL", "MongoDB", "Cassandra", "HBase", "Postgres"], ["Python", "scikit-learn", "scipy", "numpy", "statsmodels", "pandas"], ["R", "Python", "statistics", "regression", "probability"], ["machine learning", "regression", "decision trees", "libsvm"], ["Python", "R", "Java", "C++", "Haskell", "programming languages"], ["statistics", "probability", "mathematics", "theory"], ["machine learning", "scikit-learn", "Mahout", "neural networks"], ["neural networks", "deep learning", "Big Data", "artificial intelligence"], ["Hadoop", "Java", "MapReduce", "Big Data"], ["statistics", "R", "statsmodels"], ["C++", "deep learning", "artificial intelligence", "probability"], ["pandas", "R", "Python"], ["databases", "HBase", "Postgres", "MySQL", "MongoDB"], ["libsvm", "regression", "support vector machines"] ] And we’ll try to find K = 4 topics. In order to calculate the sampling weights, we’ll need to keep track of several counts.

…

In particular, we’ll look at the data set of users_interests that we’ve used before: users_interests = [ ["Hadoop", "Big Data", "HBase", "Java", "Spark", "Storm", "Cassandra"], ["NoSQL", "MongoDB", "Cassandra", "HBase", "Postgres"], ["Python", "scikit-learn", "scipy", "numpy", "statsmodels", "pandas"], ["R", "Python", "statistics", "regression", "probability"], ["machine learning", "regression", "decision trees", "libsvm"], ["Python", "R", "Java", "C++", "Haskell", "programming languages"], ["statistics", "probability", "mathematics", "theory"], ["machine learning", "scikit-learn", "Mahout", "neural networks"], ["neural networks", "deep learning", "Big Data", "artificial intelligence"], ["Hadoop", "Java", "MapReduce", "Big Data"], ["statistics", "R", "statsmodels"], ["C++", "deep learning", "artificial intelligence", "probability"], ["pandas", "R", "Python"], ["databases", "HBase", "Postgres", "MySQL", "MongoDB"], ["libsvm", "regression", "support vector machines"] ] And we’ll think about the problem of recommending new interests to a user based on her currently specified interests.

…

After asking around, you manage to get your hands on this data, as a list of pairs (user_id, interest): interests = [ (0, "Hadoop"), (0, "Big Data"), (0, "HBase"), (0, "Java"), (0, "Spark"), (0, "Storm"), (0, "Cassandra"), (1, "NoSQL"), (1, "MongoDB"), (1, "Cassandra"), (1, "HBase"), (1, "Postgres"), (2, "Python"), (2, "scikit-learn"), (2, "scipy"), (2, "numpy"), (2, "statsmodels"), (2, "pandas"), (3, "R"), (3, "Python"), (3, "statistics"), (3, "regression"), (3, "probability"), (4, "machine learning"), (4, "regression"), (4, "decision trees"), (4, "libsvm"), (5, "Python"), (5, "R"), (5, "Java"), (5, "C++"), (5, "Haskell"), (5, "programming languages"), (6, "statistics"), (6, "probability"), (6, "mathematics"), (6, "theory"), (7, "machine learning"), (7, "scikit-learn"), (7, "Mahout"), (7, "neural networks"), (8, "neural networks"), (8, "deep learning"), (8, "Big Data"), (8, "artificial intelligence"), (9, "Hadoop"), (9, "Java"), (9, "MapReduce"), (9, "Big Data") ] For example, Thor (id 4) has no friends in common with Devin (id 7), but they share an interest in machine learning. It’s easy to build a function that finds users with a certain interest: def data_scientists_who_like(target_interest): return [user_id for user_id, user_interest in interests if user_interest == target_interest] This works, but it has to examine the whole list of interests for every search.

Rise of the Robots: Technology and the Threat of a Jobless Future by Martin Ford

3D printing, additive manufacturing, Affordable Care Act / Obamacare, AI winter, algorithmic management, algorithmic trading, Amazon Mechanical Turk, artificial general intelligence, assortative mating, autonomous vehicles, banking crisis, basic income, Baxter: Rethink Robotics, Bernie Madoff, Bill Joy: nanobots, bond market vigilante , business cycle, call centre, Capital in the Twenty-First Century by Thomas Piketty, carbon tax, Charles Babbage, Chris Urmson, Clayton Christensen, clean water, cloud computing, collateralized debt obligation, commoditize, computer age, creative destruction, data science, debt deflation, deep learning, deskilling, digital divide, disruptive innovation, diversified portfolio, driverless car, Erik Brynjolfsson, factory automation, financial innovation, Flash crash, Ford Model T, Fractional reserve banking, Freestyle chess, full employment, general purpose technology, Geoffrey Hinton, Goldman Sachs: Vampire Squid, Gunnar Myrdal, High speed trading, income inequality, indoor plumbing, industrial robot, informal economy, iterative process, Jaron Lanier, job automation, John Markoff, John Maynard Keynes: technological unemployment, John von Neumann, Kenneth Arrow, Khan Academy, Kiva Systems, knowledge worker, labor-force participation, large language model, liquidity trap, low interest rates, low skilled workers, low-wage service sector, Lyft, machine readable, machine translation, manufacturing employment, Marc Andreessen, McJob, moral hazard, Narrative Science, Network effects, new economy, Nicholas Carr, Norbert Wiener, obamacare, optical character recognition, passive income, Paul Samuelson, performance metric, Peter Thiel, plutocrats, post scarcity, precision agriculture, price mechanism, public intellectual, Ray Kurzweil, rent control, rent-seeking, reshoring, RFID, Richard Feynman, Robert Solow, Rodney Brooks, Salesforce, Sam Peltzman, secular stagnation, self-driving car, Silicon Valley, Silicon Valley billionaire, Silicon Valley startup, single-payer health, software is eating the world, sovereign wealth fund, speech recognition, Spread Networks laid a new fibre optics cable between New York and Chicago, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Steven Levy, Steven Pinker, strong AI, Stuxnet, technological singularity, telepresence, telepresence robot, The Bell Curve by Richard Herrnstein and Charles Murray, The Coming Technological Singularity, The Future of Employment, the long tail, Thomas L Friedman, too big to fail, Tragedy of the Commons, Tyler Cowen, Tyler Cowen: Great Stagnation, uber lyft, union organizing, Vernor Vinge, very high income, warehouse automation, warehouse robotics, Watson beat the top human players on Jeopardy!, women in the workforce

However, the last few years have seen a number of dramatic breakthroughs that have resulted in significant advances in performance, especially when multiple layers of neurons are employed—a technology that has come to be called “deep learning.” Deep learning systems already power the speech recognition capability in Apple’s Siri and are poised to accelerate progress in a broad range of applications that rely on pattern analysis and recognition. A deep learning neural network designed in 2011 by scientists at the University of Lugano in Switzerland, for example, was able to correctly identify more than 99 percent of the images in a large database of traffic signs—a level of accuracy that exceeded that of human experts who competed against the system.

…

That compares with 97.53 percent accuracy for human observers.9 Geoffrey Hinton of the University of Toronto, one of the leading researchers in the field, notes that deep learning technology “scales beautifully. Basically you just need to keep making it bigger and faster, and it will get better.”10 In other words, even without accounting for likely future improvements in their design, machine learning systems powered by deep learning networks are virtually certain to see continued dramatic progress simply as a result of Moore’s Law. Big data and the smart algorithms that accompany it are having an immediate impact on workplaces and careers as employers, particularly large corporations, increasingly track a myriad of metrics and statistics regarding the work and social interactions of their employees.

…

Tom Simonite, “Facebook Creates Software That Matches Faces Almost as Well as You Do,” MIT Technology Review, March 17, 2014, http://www.technologyreview.com/news/525586/facebook-creates-software-that-matches-faces-almost-as-well-as-you-do/. 10. As quoted in John Markoff, “Scientists See Promise in Deep-Learning Programs,” New York Times, November 23, 2012, http://www.nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html. 11. Don Peck, “They’re Watching You at Work,” The Atlantic, December 2013, http://www.theatlantic.com/magazine/archive/2013/12/theyre-watching-you-at-work/354681/. 12. United States Patent No. 8,589,407, “Automated Generation of Suggestions for Personalized Reactions in a Social Network,” November 19, 2013, http://patft.uspto.gov/netacgi/nph-Parser?

From Bacteria to Bach and Back: The Evolution of Minds by Daniel C. Dennett

Ada Lovelace, adjacent possible, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, AlphaGo, Andrew Wiles, Bayesian statistics, bioinformatics, bitcoin, Bletchley Park, Build a better mousetrap, Claude Shannon: information theory, computer age, computer vision, Computing Machinery and Intelligence, CRISPR, deep learning, disinformation, double entry bookkeeping, double helix, Douglas Hofstadter, Elon Musk, epigenetics, experimental subject, Fermat's Last Theorem, Gödel, Escher, Bach, Higgs boson, information asymmetry, information retrieval, invention of writing, Isaac Newton, iterative process, John von Neumann, language acquisition, megaproject, Menlo Park, Murray Gell-Mann, Necker cube, Norbert Wiener, pattern recognition, phenotype, Richard Feynman, Rodney Brooks, self-driving car, social intelligence, sorting algorithm, speech recognition, Stephen Hawking, Steven Pinker, strong AI, Stuart Kauffman, TED Talk, The Wealth of Nations by Adam Smith, theory of mind, Thomas Bayes, trickle-down economics, Turing machine, Turing test, Watson beat the top human players on Jeopardy!, Y2K

This suggests—but certainly does not prove—that without us machine users to interpret the results, critically and insightfully, deep-learning machines may grow in competence, surpassing animal brains (including ours) by orders of magnitude in the bottom-up task of finding statistical regularities, but never achieve (our kind of) comprehension. “So what?” some might respond. “The computer kind of bottom-up comprehension will eventually submerge the human kind, overpowering it with the sheer size and speed of its learning.” The latest breakthrough in AI, AlphaGo, the deep-learning program that has recently beaten Lee Seedol, regarded by many as the best human player of Go in the world, supports this expectation in one regard if not in others.

…

Dehaene and Naccache (2001) note “the impossibility for subjects [i.e., executives] to strategically use the unconscious information.” My claim, then, is that deep learning (so far) discriminates but doesn’t notice. That is, the flood of data that a system takes in does not have relevance for the system except as more “food” to “digest.” Being bedridden, not having to fend for itself, it has no goals beyond increasing its store of well-indexed information. Beyond the capacity we share with Watson and other deep learning machines to acquire know-how that depends on statistical regularities that we extract from experience, there is the capacity to decide what to search for and why, given one’s current aims.

…

A conscious human mind is not a miracle, not a violation of the principles of natural selection, but a novel extension of them, a new crane that adjusts evolutionary biologist Stuart Kauffman’s concept of the adjacent possible: many more places in Design Space are adjacent to us because we have evolved the ability to think about them and either seek them or shun them. The unanswered question for Domingos and other exponents of deep learning is whether learning a sufficiently detailed and dynamic theory of agents with imagination and reason-giving capabilities would enable a system (a computer program, a Master Algorithm) to generate and exploit the abilities of such agents, that is to say, to generate all the morally relevant powers of a person.103 My view is (still) that deep learning will not give us—in the next fifty years—anything like the “superhuman intelligence” that has attracted so much alarmed attention recently (Bostrom 2014; earlier invocations are Moravec 1988; Kurzweil 2005; and Chalmers 2010; see also the annual Edge world question 2015; and Katchadourian 2015).

Army of None: Autonomous Weapons and the Future of War by Paul Scharre

"World Economic Forum" Davos, active measures, Air France Flight 447, air gap, algorithmic trading, AlphaGo, Apollo 13, artificial general intelligence, augmented reality, automated trading system, autonomous vehicles, basic income, Black Monday: stock market crash in 1987, brain emulation, Brian Krebs, cognitive bias, computer vision, cuban missile crisis, dark matter, DARPA: Urban Challenge, data science, deep learning, DeepMind, DevOps, Dr. Strangelove, drone strike, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, facts on the ground, fail fast, fault tolerance, Flash crash, Freestyle chess, friendly fire, Herman Kahn, IFF: identification friend or foe, ImageNet competition, information security, Internet of things, Jeff Hawkins, Johann Wolfgang von Goethe, John Markoff, Kevin Kelly, Korean Air Lines Flight 007, Loebner Prize, loose coupling, Mark Zuckerberg, military-industrial complex, moral hazard, move 37, mutually assured destruction, Nate Silver, Nick Bostrom, PalmPilot, paperclip maximiser, pattern recognition, Rodney Brooks, Rubik’s Cube, self-driving car, sensor fusion, South China Sea, speech recognition, Stanislav Petrov, Stephen Hawking, Steve Ballmer, Steve Wozniak, Strategic Defense Initiative, Stuxnet, superintelligent machines, Tesla Model S, The Signal and the Noise by Nate Silver, theory of mind, Turing test, Tyler Cowen, universal basic income, Valery Gerasimov, Wall-E, warehouse robotics, William Langewiesche, Y2K, zero day

Grainy SAR images of tanks, artillery, or airplanes parked on a runway often push the limits of human abilities to recognize objects, and historically ATR algorithms have fallen far short of human abilities. The poor performance of military ATR stands in stark contrast to recent advances in computer vision. Artificial intelligence has historically struggled with object recognition and perception, but the field has seen rapid gains recently due to deep learning. Deep learning uses neural networks, a type of AI approach that is analogous to biological neurons in animal brains. Artificial neural networks don’t directly mimic biology, but are inspired by it. Rather than follow a script of if-then steps for how to perform a task, neural networks work based on the strength of connections within a network.

…

Thus the response of the network to all possible inputs is unknowable. Part of this is due to the early stage of research in neural nets, but part of it is due to the sheer complexity of the deep learning. The JASON group argued that “the very nature of [deep neural networks] may make it intrinsically difficult for them to transition into what is typically recognized as a professionally engineered product.” AI researchers are working on ways to build more transparent AI, but Jeff Clune isn’t hopeful. “As deep learning gets even more powerful and more impressive and more complicated and as the networks grow in size, there will be more and more and more things we don’t understand. . . .

…

Goodfellow, Jonathan Shlens, and Christian Szegedy, “Explaining and Harnessing Adversarial Examples,” March 20, 2015, https://arxiv.org/abs/1412.6572; Ian Goodfellow, Presentation at Re-Work Deep Learning Summit, 2015, https://www.youtube.com/watch?v=Pq4A2mPCB0Y. 184 “infinitely far to the left”: Jeff Clune, interview, September 28, 2016. 184 “real-world images are a very, very small”: Ibid. 184 present in essentially every deep neural network: “Deep neural networks are easily fooled.” Goodfellow et al., “Explaining and Harnessing Adversarial Examples.” 184 specially evolved noise: Corey Kereliuk, Bob L. Sturm, and Jan Larsen, “Deep Learning and Music Adversaries,” http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/6904/pdf/imm6904.pdf. 184 News-reading trading bots: John Carney, “The Trading Robots Really Are Reading Twitter,” April 23, 2013, http://www.cnbc.com/id/100666302.

Life After Google: The Fall of Big Data and the Rise of the Blockchain Economy by George Gilder

23andMe, Airbnb, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, Albert Einstein, AlphaGo, AltaVista, Amazon Web Services, AOL-Time Warner, Asilomar, augmented reality, Ben Horowitz, bitcoin, Bitcoin Ponzi scheme, Bletchley Park, blockchain, Bob Noyce, British Empire, Brownian motion, Burning Man, business process, butterfly effect, carbon footprint, cellular automata, Claude Shannon: information theory, Clayton Christensen, cloud computing, computer age, computer vision, crony capitalism, cross-subsidies, cryptocurrency, Danny Hillis, decentralized internet, deep learning, DeepMind, Demis Hassabis, disintermediation, distributed ledger, don't be evil, Donald Knuth, Donald Trump, double entry bookkeeping, driverless car, Elon Musk, Erik Brynjolfsson, Ethereum, ethereum blockchain, fake news, fault tolerance, fiat currency, Firefox, first square of the chessboard, first square of the chessboard / second half of the chessboard, floating exchange rates, Fractional reserve banking, game design, Geoffrey Hinton, George Gilder, Google Earth, Google Glasses, Google Hangouts, index fund, inflation targeting, informal economy, initial coin offering, Internet of things, Isaac Newton, iterative process, Jaron Lanier, Jeff Bezos, Jim Simons, Joan Didion, John Markoff, John von Neumann, Julian Assange, Kevin Kelly, Law of Accelerating Returns, machine translation, Marc Andreessen, Mark Zuckerberg, Mary Meeker, means of production, Menlo Park, Metcalfe’s law, Money creation, money: store of value / unit of account / medium of exchange, move fast and break things, Neal Stephenson, Network effects, new economy, Nick Bostrom, Norbert Wiener, Oculus Rift, OSI model, PageRank, pattern recognition, Paul Graham, peer-to-peer, Peter Thiel, Ponzi scheme, prediction markets, quantitative easing, random walk, ransomware, Ray Kurzweil, reality distortion field, Recombinant DNA, Renaissance Technologies, Robert Mercer, Robert Metcalfe, Ronald Coase, Ross Ulbricht, Ruby on Rails, Sand Hill Road, Satoshi Nakamoto, Search for Extraterrestrial Intelligence, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, Singularitarianism, Skype, smart contracts, Snapchat, Snow Crash, software is eating the world, sorting algorithm, South Sea Bubble, speech recognition, Stephen Hawking, Steve Jobs, Steven Levy, Stewart Brand, stochastic process, Susan Wojcicki, TED Talk, telepresence, Tesla Model S, The Soul of a New Machine, theory of mind, Tim Cook: Apple, transaction costs, tulip mania, Turing complete, Turing machine, Vernor Vinge, Vitalik Buterin, Von Neumann architecture, Watson beat the top human players on Jeopardy!, WikiLeaks, Y Combinator, zero-sum game

CHAPTER 3 Google’s Roots and Religions Under the leadership of Larry Page and Sergey Brin, Google developed the integrated philosophy that currently shapes our lives and fortunes, combining a theory of knowledge (nicknamed “Big Data”), a technological vision (centralized cloud computing), a cult of the commons (rooted in “open source” software), a concept of money and value (based on free goods and automated advertising), a theory of morality as “gifts” rather than profits, and a view of progress as evolutionary inevitability and an ever diminishing “carbon footprint.” This philosophy rules our economic lives in America and, increasingly, around the globe. With its development of “deep learning” by machines and its hiring of the inventor-prophet Raymond Kurzweil in 2014, Google enlisted in a chiliastic campaign to blend human and machine cognition. Kurzweil calls it a “singularity,” marked by the triumph of computation over human intelligence. Google networks, clouds, and server farms could be said to have already accomplished much of it.

…

Google, meanwhile, under its new CEO, Sundar Pichai, pivoted away from its highly publicized “mobile first” mantra, which had led to its acquisitions of Android and Ad Mob, and toward “AI first.” Google was the recognized intellectual leader of the industry, and its AI ostentation was widely acclaimed. Indeed it signed up most of the world’s AI celebrities, including its spearheads of “deep learning” prowess, from Geoffrey Hinton and Andrew Ng to Jeff Dean, the beleaguered Anthony Levandowski, and Demis Hassabis of DeepMind. If Google had been a university, it would have utterly outshone all others in AI talent. It must have been discouraging, then, to find that Amazon had shrewdly captured much of the market for AI services with its 2014 Alexa and Echo projects.

…

Or do you diffuse the memory and processing all through the machine? In a massively parallel spread like Dally’s J-machine, the memory is always close to the processor. Twenty-six years later, Dally and Jouppi are still at it. At the August 2017 Hot Chips in Cupertino, all the big guys were touting their own chips for what they call “deep learning,” the fashionable Silicon Valley term for the massive acceleration of multi-layered pattern recognition, correlation, and correction tied to feedback that results in a cumulative gain in performance. What they call “learning” originated in earlier ventures in AI. Guess, measure the error, adjust the answer, feed it back are the canonical steps followed in Google’s data centers, enabling such applications as Google Translate, Google Soundwriter, Google Maps, Google Assistant, Waymo cars, search, Google Now, and so on, in real time.4 As recently as 2012, Google was still struggling with the difference between dogs and cats.

Human + Machine: Reimagining Work in the Age of AI by Paul R. Daugherty, H. James Wilson

3D printing, AI winter, algorithmic management, algorithmic trading, AlphaGo, Amazon Mechanical Turk, Amazon Robotics, augmented reality, autonomous vehicles, blockchain, business process, call centre, carbon footprint, circular economy, cloud computing, computer vision, correlation does not imply causation, crowdsourcing, data science, deep learning, DeepMind, digital twin, disintermediation, Douglas Hofstadter, driverless car, en.wikipedia.org, Erik Brynjolfsson, fail fast, friendly AI, fulfillment center, future of work, Geoffrey Hinton, Hans Moravec, industrial robot, Internet of things, inventory management, iterative process, Jeff Bezos, job automation, job satisfaction, knowledge worker, Lyft, machine translation, Marc Benioff, natural language processing, Neal Stephenson, personalized medicine, precision agriculture, Ray Kurzweil, recommendation engine, RFID, ride hailing / ride sharing, risk tolerance, robotic process automation, Rodney Brooks, Salesforce, Second Machine Age, self-driving car, sensor fusion, sentiment analysis, Shoshana Zuboff, Silicon Valley, Snow Crash, software as a service, speech recognition, tacit knowledge, telepresence, telepresence robot, text mining, the scientific method, uber lyft, warehouse automation, warehouse robotics

These individuals are responsible for making important judgment calls about which AI technologies might best be deployed for specific applications. A huge consideration here is accuracy versus “explainability.” A deep-learning system, for example, provides a high level of prediction accuracy, but companies may have difficulty explaining how those results were derived. In contrast, a decision tree may not lead to results with high prediction accuracy but will enable a significantly greater explainability. So, for instance, an internal system that optimizes a supply chain with small tolerances for scheduling deliveries might best deploy deep-learning technology, whereas a health-care or consumer-facing application that will have to stand up to considerable regulatory scrutiny may be better off utilizing falling rule list algorithms.10 In addition, the explainability strategist might also decide that, for a particular application, the company might be better off avoiding the use of AI altogether.

…

See personalization cybersecurity, 56–58, 59 Darktrace, 58 DARPA Cyber Grand Challenges, 57, 190 Dartmouth College conference, 40–41 dashboards, 169 data, 10 in AI training, 121–122 barriers to flow of, 176–177 customization and, 78–80 discovery with, 178 dynamic, real-time, 175–176 in enterprise processes, 59 exhaust, 15 in factories, 26–27, 29–30 leadership and, 180 in manufacturing, 38–39 in marketing and sales, 92, 98–99, 100 in R&D, 69–72 in reimagining processes, 154 on supply chains, 33–34 supply chains for, 12, 15 velocity of, 177–178 data hygienists, 121–122 data supply-chain officers, 179 data supply chains, 12, 15, 174–179 decision making, 109–110 about brands, 93–94 black box, 106, 125, 169 employee power to modify AI, 172–174 empowerment for, 15 explainers and, 123–126 transparency in, 213 Deep Armor, 58 deep learning, 63, 161–165 deep-learning algorithms, 125 DeepMind, 121 deep neural networks (DNN), 63 deep reinforcement learning, 21–22 demand planning, 33–34 Dennis, Jamie, 158 design at Airbus, 144 AI system, 128–129 Elbo Chair, 135–137 generative, 135–137, 139, 141 product/service, 74–77 Dickey, Roger, 52–54 digital twins, 10 at GE, 27, 29–30, 183–184, 194 disintermediation, brand, 94–95 distributed learning, 22 distribution, 19–39 Ditto Labs, 98 diversity, 52 Doctors Without Borders, 151 DoubleClick Search, 99 Dreamcatcher, 136–137, 141, 144 drones, 28, 150–151 drug interactions, 72–74 Ducati, 175 Echo, 92, 164–165 Echo Voyager, 28 Einstein, 85–86, 196 Elbo Chair, 136–137, 139 “Elephants Don’t Play Chess” (Brooks), 24 Elish, Madeleine Clare, 170–171 Ella, 198–199 embodied intelligence, 206 embodiment, 107, 139–140 in factories, 21–23 of intelligence, 206 interaction agents, 146–151 jobs with, 147–151 See also augmentation; missing middle empathy engines for health care, 97 training, 117–118, 132 employees agency of, 15, 172–174 amplification of, 138–139, 141–143 development of, 14 hiring, 51–52 job satisfaction in, 46–47 marketing and sales, 90, 92, 100–101 on-demand work and, 111 rehumanizing time and, 186–189 routine/repetitive work and, 26–27, 29–30, 46–47 training/retraining, 15 warehouse, 31–33 empowerment, 137 bot-based, 12, 195–196 in decision making, 15 of salespeople, 90, 92 workforce implications of, 137–138 enabling, 7 enterprise processes, 45–66 compliance, 47–48 determining which to change, 52–54 hiring and recruitment, 51–52 how much to change, 54–56 redefining industries with, 56–58 reimagining around people, 58–59 robotic process automation (RPA) in, 50–52 routine/repetitive, 46–47 ergonomics, 149–150 EstherBot, 199 ethical, moral, legal issues, 14–15, 108 Amazon Echo and, 164–165 explainers and, 123–126 in marketing and sales, 90, 100 moral crumple zones and, 169–172 privacy, 90 in R&D, 83 in research, 78–79 ethics compliance managers, 79, 129–130, 132–133 European Union, 124 Ewing, Robyn, 119 exhaust data, 15 definition of, 122 experimentation, 12, 14 cultures of, 161–165 in enterprise processes, 59 leadership and, 180 learning from, 71 in manufacturing, 39 in marketing and sales, 100 in process reimagining, 160–165 in R&D, 83 in reimagining processes, 154 testing and, 74–77 expert systems, 25, 41 definition of, 64 explainability strategists, 126 explaining outcomes, 107, 114–115, 179 black-box concerns and, 106, 125, 169 jobs in, 122–126 sustaining and, 130 See also missing middle extended intelligence, 206 extended reality, 66 Facebook, 78, 79, 95, 177–178 facial recognition, 65, 90 factories, 10 data flow in, 26–27, 29–30 embodiment in, 140 job losses and gains in, 19, 20 robotic arms in, 21–26 self-aware, 19–39 supply chains and, 33–34 third wave in, 38–39 traditional assembly lines and, 1–2, 4 warehouse management and, 30–33 failure, learning from, 71 fairness, 129–130 falling rule list algorithms, 124–125 Fanuc, 21–22, 128 feedback, 171–172 feedforward neural networks (FNN), 63 Feigenbaum, Ed, 41 financial trading, 167 first wave of business transformation, 5 Fletcher, Seth, 49 food production, 34–37 ForAllSecure, 57 forecasts, 33–34 Fortescue Metals Group, 28 Fraunhofer Institute of Material Flow and Logistics (IML), 26 fusion skills, 12, 181, 183–206, 210 bot-based empowerment, 12, 195–196 developing, 15–16 holistic melding, 12, 197, 200–201 intelligent interrogation, 12, 185, 193–195 judgment integration, 12, 191–193 potential of, 209 reciprocal apprenticing, 12, 201–202 rehumanizing time, 12, 186–189 relentless reimagining, 12, 203–205 responsible normalizing, 12, 189–191 training/retraining for, 211–213 Future of Work survey, 184–185 Garage, Capital One, 205 Gaudin, Sharon, 99 GE.

…

From the Mechanistic to the Organic The potential power of AI to transform businesses is unprecedented, and yet there is an urgent and growing challenge. Companies are now reaching a crossroad in their use of AI, which we define as systems that extend human capability by sensing, comprehending, acting, and learning. As businesses deploy such systems—spanning from machine learning to computer vision to deep learning—some firms will continue to see modest productivity gains over the short run, but those results will eventually stall out. Other companies will be able to attain breakthrough improvements in performance, often by developing game-changing innovations. What accounts for the difference? It has to do with understanding the true nature of AI’s impact.

Finding Alphas: A Quantitative Approach to Building Trading Strategies by Igor Tulchinsky

algorithmic trading, asset allocation, automated trading system, backpropagation, backtesting, barriers to entry, behavioural economics, book value, business cycle, buy and hold, capital asset pricing model, constrained optimization, corporate governance, correlation coefficient, credit crunch, Credit Default Swap, currency risk, data science, deep learning, discounted cash flows, discrete time, diversification, diversified portfolio, Eugene Fama: efficient market hypothesis, financial engineering, financial intermediation, Flash crash, Geoffrey Hinton, implied volatility, index arbitrage, index fund, intangible asset, iterative process, Long Term Capital Management, loss aversion, low interest rates, machine readable, market design, market microstructure, merger arbitrage, natural language processing, passive investing, pattern recognition, performance metric, Performance of Mutual Funds in the Period, popular capitalism, prediction markets, price discovery process, profit motive, proprietary trading, quantitative trading / quantitative finance, random walk, Reminiscences of a Stock Operator, Renaissance Technologies, risk free rate, risk tolerance, risk-adjusted returns, risk/return, selection bias, sentiment analysis, shareholder value, Sharpe ratio, short selling, Silicon Valley, speech recognition, statistical arbitrage, statistical model, stochastic process, survivorship bias, systematic bias, systematic trading, text mining, transaction costs, Vanguard fund, yield curve

Another alternative is FloatBoost, which incorporates the backtracking mechanism of floating search and repeatedly performs a backtracking to remove unfavorable weak classifiers after a new weak classifier is added by AdaBoost; this ensures a lower error rate and reduced feature set at the cost of about five times longer training time. Deep Learning Deep learning (DL) is a popular topic today – and a term that is used to discuss a number of rather distinct things. Some data scientists think DL is just a buzz word or a rebranding of neural networks. The name comes from Canadian scientist Geoffrey Hinton, who created an unsupervised method known as the restricted Boltzmann machine (RBM) for pretraining NNs with a large number of neuron layers. That was meant to improve on the backpropagation training method, but there is no strong evidence that it really was an improvement. Another direction in deep learning is recurrent neural networks (RNNs) and natural language processing.

…

This is called the vanishing gradient problem. These days, the words “deep learning” more often refer to convolutional neural networks (CNNs). The architecture of CNNs was introduced by computer scientists Kunihiko Fukushima, who developed the 126 Finding Alphas neocognitron model (feed-forward NN), and Yann LeCun, who modified the backpropagation algorithm for neocognitron training. CNNs require a lot of resources for training, but they can be easily parallelized and therefore are a good candidate for parallel computations. When applying deep learning, we seek to stack several independent neural network layers that by working together produce better results than the shallow individual structures.

…

For example, in alpha research the task of predicting stock prices can be a good application of supervised learning, and the task of selecting stocks for inclusion in a portfolio is an application of unsupervised learning. Machine Learning in Alpha Research123 Machine learning Unsupervised methods Clusterization algorithms Supervised methods Statistical models Support vector machines Neural networks Deep learning algorithms Fuzzy logic Ensemble methods Random forest AdaBoost Figure 16.1 The most developed directions of machine learning. The most popular are in black Statistical Models Models like naive Bayes, linear discriminant analysis, the hidden Markov model, and logistic regression are good for solving relatively simple problems that do not need high precision of classification or prediction.

Heart of the Machine: Our Future in a World of Artificial Emotional Intelligence by Richard Yonck

3D printing, AI winter, AlphaGo, Apollo 11, artificial general intelligence, Asperger Syndrome, augmented reality, autism spectrum disorder, backpropagation, Berlin Wall, Bletchley Park, brain emulation, Buckminster Fuller, call centre, cognitive bias, cognitive dissonance, computer age, computer vision, Computing Machinery and Intelligence, crowdsourcing, deep learning, DeepMind, Dunning–Kruger effect, Elon Musk, en.wikipedia.org, epigenetics, Fairchild Semiconductor, friendly AI, Geoffrey Hinton, ghettoisation, industrial robot, Internet of things, invention of writing, Jacques de Vaucanson, job automation, John von Neumann, Kevin Kelly, Law of Accelerating Returns, Loebner Prize, Menlo Park, meta-analysis, Metcalfe’s law, mirror neurons, Neil Armstrong, neurotypical, Nick Bostrom, Oculus Rift, old age dependency ratio, pattern recognition, planned obsolescence, pneumatic tube, RAND corporation, Ray Kurzweil, Rodney Brooks, self-driving car, Skype, social intelligence, SoftBank, software as a service, SQL injection, Stephen Hawking, Steven Pinker, superintelligent machines, technological singularity, TED Talk, telepresence, telepresence robot, The future is already here, The Future of Employment, the scientific method, theory of mind, Turing test, twin studies, Two Sigma, undersea cable, Vernor Vinge, Watson beat the top human players on Jeopardy!, Whole Earth Review, working-age population, zero day

These processors made it possible to speed up the network training by orders of magnitude, performing the major number crunching required to reduce what once took weeks to a matter of days or hours. Different approaches led to further refining of these deep learning techniques, using methods with names such as restricted Boltzmann machines and recurrent neural networks. All of these factors vastly improved the deep learning algorithms being used for many kinds of pattern recognition work. Continuing advances contributed to the significant gains seen by artificial intelligence during this past decade, including Facebook’s development of DeepFace, which identifies human faces in images with 97 percent accuracy.

…

Continuing advances contributed to the significant gains seen by artificial intelligence during this past decade, including Facebook’s development of DeepFace, which identifies human faces in images with 97 percent accuracy. In 2012, a University of Toronto artificial intelligence team made up of Hinton and two of his students won the annual ImageNet Large Scale Visual Recognition Competition with a deep learning neural network that blew the competition away.5 More recently, Google DeepMind used deep learning to develop the Go-playing AI, AlphaGo, training it by using a database of thirty million recorded moves from expert-level games. In March 2016, AlphaGo beat the world Go grandmaster, Lee Sedol, in four out of five games. Playing Go is considered a much bigger AI challenge than playing chess.

…

The camera technology wasn’t strong enough for us to actually measure the microexpressions that are on people’s faces, those subconscious reactions that show up on the muscles on our face before we can shut it down with our consciousness, because they’re impulse.” Continuing, Denman notes that the processing power is also now available to us to be able to run the deep learning neural networks needed to make this happen. Because of this, dozens of companies are entering the space, focusing not only on facial information but on the other ways we engage the world emotionally. Tel Aviv–based Beyond Verbal is an emotions analytics company that extracts and identifies feelings conveyed by intonations in the human voice.

The Price of Tomorrow: Why Deflation Is the Key to an Abundant Future by Jeff Booth

3D printing, Abraham Maslow, activist fund / activist shareholder / activist investor, additive manufacturing, AI winter, Airbnb, Albert Einstein, AlphaGo, Amazon Web Services, artificial general intelligence, augmented reality, autonomous vehicles, basic income, bitcoin, blockchain, Bretton Woods, business intelligence, butterfly effect, Charles Babbage, Claude Shannon: information theory, clean water, cloud computing, cognitive bias, collapse of Lehman Brothers, Computing Machinery and Intelligence, corporate raider, creative destruction, crony capitalism, crowdsourcing, cryptocurrency, currency manipulation / currency intervention, dark matter, deep learning, DeepMind, deliberate practice, digital twin, distributed ledger, Donald Trump, Elon Musk, fiat currency, Filter Bubble, financial engineering, full employment, future of work, game design, gamification, general purpose technology, Geoffrey Hinton, Gordon Gekko, Great Leap Forward, Hyman Minsky, hype cycle, income inequality, inflation targeting, information asymmetry, invention of movable type, Isaac Newton, Jeff Bezos, John Maynard Keynes: Economic Possibilities for our Grandchildren, John von Neumann, Joseph Schumpeter, late fees, low interest rates, Lyft, Maslow's hierarchy, Milgram experiment, Minsky moment, Modern Monetary Theory, moral hazard, Nelson Mandela, Network effects, Nick Bostrom, oil shock, OpenAI, pattern recognition, Ponzi scheme, quantitative easing, race to the bottom, ride hailing / ride sharing, self-driving car, software as a service, technoutopianism, TED Talk, the long tail, the scientific method, Thomas Bayes, Turing test, Uber and Lyft, uber lyft, universal basic income, winner-take-all economy, X Prize, zero-sum game

When artificial intelligence reduces that waste and increases the benefits to society, as a by-product of removing the waste in the system, it reduces the number of jobs in healthcare. With more than $3.5 trillion of annual spending and 19 percent of the US GDP in healthcare, that could mean a lot of jobs. For instance, as reported in a May 2019 Nature Medicine article, researchers created a 3D volumetric deep learning model to screen for lung cancer.55 When comparing a single image, the deep learning model outperformed six radiology experts, with an 11 percent reduction in false positives and a 5 percent reduction in false negatives. According to Dr. Mozziyar Etemadi, one of the study’s coauthors, “AI in 3D can be much more sensitive in its ability to detect early lung cancer than the human eye looking at 2D images.

…

All of that digitization is also creating some impressive data capture, much more than we are even aware of, and the data collection from connected computers, people, cameras, and sensors has only just started. Connecting those devices to learn from data is arguably a far easier job than that of building the original network. The rate of growth in today’s deep learning in artificial intelligence is largely driven by data collection and large data sets. In fact, every platform company today is really a data company with AI at its core. Other data, too, is moving out of its previous silos, giving rise to an intelligence that can be combined with other data sets to learn at a rate far faster than humans.

…

Until 2014, even top AI researchers believed top human competitors would beat computers for years to come because of the complexity of the game and the fact that algorithms had to compare every move, which required enormous compute power. But in 2016, Google’s DeepMind program AlphaGo beat one of the top players in the world, Lee Sedol, in a match that made history. AlphaGo’s program was based on deep learning, which was “trained” using thousands of human amateur and professional games. It made history not only because it was the first time a computer beat a top Go master, but also because of the way it did so. In game 2 and the thirty-seventh move, the computer made a move that defied logic, placing a black stone in the middle of an open area—away from the other stones.

Explaining Humans: What Science Can Teach Us About Life, Love and Relationships by Camilla Pang

autism spectrum disorder, backpropagation, bioinformatics, Brownian motion, correlation does not imply causation, data science, deep learning, driverless car, frictionless, job automation, John Nash: game theory, John von Neumann, Kickstarter, Nash equilibrium, neurotypical, phenotype, random walk, self-driving car, stem cell, Stephen Hawking

Compared to traditional machine learning, a neural network is more independent and requires less input from the programmer to define what it should be searching for, since, through internal layers of logic, it is able to create its own connections. All of the more radical examples of artificial intelligence you may have read about – from fully driverless cars to mass automation of people’s jobs – ultimately rely on deep learning, the closest we have so far got to developing a computer program that can think (within considerable limitations) like a human. Deep learning is also responsible for applications, including criminal checks, drug design, and the computer programs that rival the most competent chess players, all of which depend on an ability to simulate the connective capability of the human mind.

…

How not to follow the crowd Molecular dynamics, conformity and individuality 7. How to achieve your goals Quantum physics, network theory and goal setting 8. How to have empathy with others Evolution, probability and relationships 9. How to connect with others Chemical bonds, fundamental forces and human connection 10. How to learn from your mistakes Deep learning, feedback loops and human memory 11. How to be polite Game theory, complex systems and etiquette Afterword Acknowledgements Index About the Author Dr Camilla Pang holds a PhD in Biochemistry from University College London and is a Postdoctoral Scientist specialising in Translational Bioinformatics.

…

No one can ever be entirely dispassionate, objective, or dare I say scientific, about how they form new relationships. But chemistry can give us a new outlook and a fresh perspective: one that provides the confidence to make, break and sometimes re-make the connections that define us. 10. How to learn from your mistakes Deep learning, feedback loops and human memory With ADHD, you’re always forgetting what you’re meant to be doing. My working memory – the part where we hold information for short-term, immediate use – is constantly undermined by new thoughts, impulses or emotional responses. It feels as though everywhere you go, even just to the room next door, the working memory is always being refreshed, losing immediate context.

The AI-First Company by Ash Fontana

23andMe, Amazon Mechanical Turk, Amazon Web Services, autonomous vehicles, barriers to entry, blockchain, business intelligence, business process, business process outsourcing, call centre, Charles Babbage, chief data officer, Clayton Christensen, cloud computing, combinatorial explosion, computer vision, crowdsourcing, data acquisition, data science, deep learning, DevOps, en.wikipedia.org, Geoffrey Hinton, independent contractor, industrial robot, inventory management, John Conway, knowledge economy, Kubernetes, Lean Startup, machine readable, minimum viable product, natural language processing, Network effects, optical character recognition, Pareto efficiency, performance metric, price discrimination, recommendation engine, Ronald Coase, Salesforce, single source of truth, software as a service, source of truth, speech recognition, the scientific method, transaction costs, vertical integration, yield management

Reinforcement learning is a functionally different approach to supervised and unsupervised ML. Transfer and deep learning overlap with the other types. The table below shows what might be applicable to a particular situation depending on the data at hand, required interpretability, and existing knowledge of the prediction problem. supervised unsupervised reinforcement transfer deep Learns from inputs given outputs inputs without outputs objectives inputs other layers in the network Needs training and feedback data lots of data objectives existing models lots of data and computational resources Good when data is available but the algorithm is missing it’s unclear what is being looked at and/or there are no labels it’s possible to articulate the state, action, reward, and how to modify the state based on the rewards problems are similar, training time and computational resources are limited, and results are needed fast there is lots of unstructured time series data (for convolutional neural networks) or data that’s not independent (for recurrent neural networks)* Selected methods include random forest trees, decision trees (including random forest and gradient boosted types), regression, support vector machines (SVMs), and neural networks clustering (k-means, hierarchical, and others) and Gaussian mixture models* various, but all forms of reinforcement learning Bayesian networks and Markov logic networks convolutional neural networks and recurrent neural networks COMPOUNDING There are many different methods for making predictions, each one generating and accumulating data in various ways.

…

Data is uploaded from the system of record, predictive models are applied, and actionable insights made available to the user. Data type Unstructured Deep learning over unstructured data to turn it into structured data and extract predictive features. There’s just not enough data about most jobs available for the training of deep learning models. Data categorization Exhaust Higher accuracy by removing reporting bias. A machine-generated summary of the last email from a sales lead is perhaps more indicative of a likelihood to close a deal with that lead than a salesperson’s subjective opinion expressed in a single, categorical input.

…

The Canadian computer scientist Yoshua Bengio devised a language model based on a neural network that figured out the next best word to use among all the available words in a language based on where that word usually appeared with respect to other words. Geoffrey Hinton, a British-born computer scientist and psychologist, developed a neural network that linked many layers of neurons together, the precursor to deep learning. Importantly, researchers worked to get these neural networks running efficiently on the available computer chips, settling on the chips used for computer graphics because they are particularly good at running many numerical computations in parallel. The result was a trainable neural network: programmable neurons, connected in a weblike network, passing the computations onto another web sitting below it—all computed on a chip that could perform the necessary operations on a reasonable timescale: mere days instead of months.

Robot Rules: Regulating Artificial Intelligence by Jacob Turner

"World Economic Forum" Davos, Ada Lovelace, Affordable Care Act / Obamacare, AI winter, algorithmic bias, algorithmic trading, AlphaGo, artificial general intelligence, Asilomar, Asilomar Conference on Recombinant DNA, autonomous vehicles, backpropagation, Basel III, bitcoin, Black Monday: stock market crash in 1987, blockchain, brain emulation, Brexit referendum, Cambridge Analytica, Charles Babbage, Clapham omnibus, cognitive dissonance, Computing Machinery and Intelligence, corporate governance, corporate social responsibility, correlation does not imply causation, crowdsourcing, data science, deep learning, DeepMind, Demis Hassabis, distributed ledger, don't be evil, Donald Trump, driverless car, easy for humans, difficult for computers, effective altruism, Elon Musk, financial exclusion, financial innovation, friendly fire, future of work, hallucination problem, hive mind, Internet of things, iterative process, job automation, John Markoff, John von Neumann, Loebner Prize, machine readable, machine translation, medical malpractice, Nate Silver, natural language processing, Nick Bostrom, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, nudge unit, obamacare, off grid, OpenAI, paperclip maximiser, pattern recognition, Peace of Westphalia, Philippa Foot, race to the bottom, Ray Kurzweil, Recombinant DNA, Rodney Brooks, self-driving car, Silicon Valley, Stanislav Petrov, Stephen Hawking, Steve Wozniak, strong AI, technological singularity, Tesla Model S, The Coming Technological Singularity, The Future of Employment, The Signal and the Noise by Nate Silver, trolley problem, Turing test, Vernor Vinge

Perez summed up these developments:So not only are researcher[s] who hand optimize gradient descent solutions out of business, so are folks who make a living designing neural architectures! This is actually just the beginning of Deep Learning systems just bootstrapping themselves… This is absolutely shocking and there’s really no end in sight as to how quickly Deep Learning algorithms are going to improve. This meta capability allows you to apply it on itself, recursively creating better and better systems.137 As noted in Chapter 1, various companies and researchers announced in 2017 that they had created AI software which could itself develop further AI software.138 In May 2017, Google demonstrated a meta-learning technology called AutoML.

…

See Cade Metz, “Google’s Dueling Neural Networks Spar to Get Smarter, No Humans Required”, Wired, 4 November 2017, https://​www.​wired.​com/​2017/​04/​googlesdueling-neural-networks-spar-get-smarter-no-humans-required/​, accessed 16 August 2018. 127Yann LeCun, “Answer to Question: What are Some Recent and Potentially Upcoming Breakthroughs in Deep Learning?”, Quora, 28 July 2016, https://​www.​quora.​com/​What-are-some-recent-and-potentially-upcoming-breakthroughs-in-deep-learning, accessed 16 August 2018. 128Andrea Bertolini, “Robots as Products: The Case for a Realistic Analysis of Robotic Applications and Liability Rules”, Law Innovation and Technology, Vol. 5, No. 2 (2013), 214–247, 234–235. 129See Chapter 1 at s. 5 and FN 111.

…

id=​rJY0-Kcll, accessed 1 June 2018. 136Andrew Ng, Jiquan Ngiam, Chuan Yu Foo, Yifan Mai, Caroline Suen, Adam Coates, Andrew Maas, Awni Hannun, Brody Huval, Tao Wang, and Sameep Tando, “Optimization: Stochastic Gradient Descent”, Stanford UFLDL Tutorial, http://​ufldl.​stanford.​edu/​tutorial/​supervised/​OptimizationStoc​hasticGradientDe​scent/​, accessed 1 June 2018. 137Carlos E. Perez, “Deep Learning: The Unreasonable Effectiveness of Randomness”, Medium, 6 November 2016, https://​medium.​com/​intuitionmachine​/​deep-learning-the-unreasonable-effectiveness-of-randomness-14d5aef13f87, accessed 1 June 2018. 138See Chapter 1 at s. 5. 139See also Sundar Pichai, “Making AI Work for Everyone”, Google Blog, 17 May 2017, https://​blog.​google/​topics/​machine-learning/​making-ai-work-for-everyone/​, accessed 1 June 2018. 140At present, many AI systems require a significant amount of human fine-tuning, especially when they are produced by companies interested in achieving striking results even at a high cost in terms of resources.

Artificial Unintelligence: How Computers Misunderstand the World by Meredith Broussard

"Susan Fowler" uber, 1960s counterculture, A Declaration of the Independence of Cyberspace, Ada Lovelace, AI winter, Airbnb, algorithmic bias, AlphaGo, Amazon Web Services, autonomous vehicles, availability heuristic, barriers to entry, Bernie Sanders, Big Tech, bitcoin, Buckminster Fuller, Charles Babbage, Chris Urmson, Clayton Christensen, cloud computing, cognitive bias, complexity theory, computer vision, Computing Machinery and Intelligence, crowdsourcing, Danny Hillis, DARPA: Urban Challenge, data science, deep learning, Dennis Ritchie, digital map, disruptive innovation, Donald Trump, Douglas Engelbart, driverless car, easy for humans, difficult for computers, Electric Kool-Aid Acid Test, Elon Musk, fake news, Firefox, gamification, gig economy, global supply chain, Google Glasses, Google X / Alphabet X, Greyball, Hacker Ethic, independent contractor, Jaron Lanier, Jeff Bezos, Jeremy Corbyn, John Perry Barlow, John von Neumann, Joi Ito, Joseph-Marie Jacquard, life extension, Lyft, machine translation, Mark Zuckerberg, mass incarceration, Minecraft, minimum viable product, Mother of all demos, move fast and break things, Nate Silver, natural language processing, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, One Laptop per Child (OLPC), opioid epidemic / opioid crisis, PageRank, Paradox of Choice, payday loans, paypal mafia, performance metric, Peter Thiel, price discrimination, Ray Kurzweil, ride hailing / ride sharing, Ross Ulbricht, Saturday Night Live, school choice, self-driving car, Silicon Valley, Silicon Valley billionaire, speech recognition, statistical model, Steve Jobs, Steven Levy, Stewart Brand, TechCrunch disrupt, Tesla Model S, the High Line, The Signal and the Noise by Nate Silver, theory of mind, traumatic brain injury, Travis Kalanick, trolley problem, Turing test, Uber for X, uber lyft, Watson beat the top human players on Jeopardy!, We are as Gods, Whole Earth Catalog, women in the workforce, work culture , yottabyte

We could make all the books available electronically and have the students access the books on their phones, because all students have phones. Wrong. Phones are great for reading short works, but long works are difficult and uncomfortable to read on a phone. Studies show that reading on a screen is worse than reading on paper in an educational context. Speed, accuracy, and deep learning all suffer when research subjects read on screens. Paper is simply a superior technology for the kind of deep learning that we want students to engage in as part of their education. Reading on a screen is fun and convenient, yes—but reading for comprehension is not about fun or convenience. It’s about learning. When it comes to learning, students generally prefer paper to screens.3 Another technochauvinist might suggest giving all the students iPads or Chromebooks or some kind of e-reader and making all the books available electronically: another good idea, but the obstacles are obvious.

…

“Reflecting on One Very, Very Strange Year at Uber.” Susan Fowler (blog), February 19, 2017. https://www.susanjfowler.com/blog/2017/2/19/reflecting-on-one-very-strange-year-at-uber. Gomes, Lee. “Facebook AI Director Yann LeCun on His Quest to Unleash Deep Learning and Make Machines Smarter.” IEEE Spectrum (blog), February 18, 2015. http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/facebook-ai-director-yann-lecun-on-deep-learning. Gray, Jonathan, Liliana Bounegru, and Lucy Chambers, eds. The Data Journalism Handbook: How Journalists Can Use Data to Improve the News. Sebastopol, CA: O’Reilly Media, 2012. Grazian, David. Mix It Up: Popular Culture, Mass Media, and Society. 2nd ed.

…

We can take the model and run new data through it to get a numerical answer that predicts something: how likely it is that a squiggle on a page is the letter A; how likely it is that a given customer will pay back the mortgage money a bank loans to him; which is the best next move to make in a game of tic-tac-toe, checkers, or chess. Machine learning, deep learning, neural networks, and predictive analytics are some of the narrow AI concepts that are currently popular. For every AI system that exists today, there is a logical explanation for how it works. Understanding the computational logic can demystify AI, just like dismantling a computer helps to demystify hardware.

Rage Inside the Machine: The Prejudice of Algorithms, and How to Stop the Internet Making Bigots of Us All by Robert Elliott Smith

"World Economic Forum" Davos, Ada Lovelace, adjacent possible, affirmative action, AI winter, Alfred Russel Wallace, algorithmic bias, algorithmic management, AlphaGo, Amazon Mechanical Turk, animal electricity, autonomous vehicles, behavioural economics, Black Swan, Brexit referendum, British Empire, Cambridge Analytica, cellular automata, Charles Babbage, citizen journalism, Claude Shannon: information theory, combinatorial explosion, Computing Machinery and Intelligence, corporate personhood, correlation coefficient, crowdsourcing, Daniel Kahneman / Amos Tversky, data science, deep learning, DeepMind, desegregation, discovery of DNA, disinformation, Douglas Hofstadter, Elon Musk, fake news, Fellow of the Royal Society, feminist movement, Filter Bubble, Flash crash, Geoffrey Hinton, Gerolamo Cardano, gig economy, Gödel, Escher, Bach, invention of the wheel, invisible hand, Jacquard loom, Jacques de Vaucanson, John Harrison: Longitude, John von Neumann, Kenneth Arrow, Linda problem, low skilled workers, Mark Zuckerberg, mass immigration, meta-analysis, mutually assured destruction, natural language processing, new economy, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, On the Economy of Machinery and Manufactures, p-value, pattern recognition, Paul Samuelson, performance metric, Pierre-Simon Laplace, post-truth, precariat, profit maximization, profit motive, Silicon Valley, social intelligence, statistical model, Stephen Hawking, stochastic process, Stuart Kauffman, telemarketer, The Bell Curve by Richard Herrnstein and Charles Murray, The Future of Employment, the scientific method, The Wealth of Nations by Adam Smith, The Wisdom of Crowds, theory of mind, Thomas Bayes, Thomas Malthus, traveling salesman, Turing machine, Turing test, twin studies, Vilfredo Pareto, Von Neumann architecture, warehouse robotics, women in the workforce, Yochai Benkler

Avoiding the stretched metaphors of neural networks, the mathematical functions represented now look like this: The typical word used to describe these networks is ‘deep’ and the weight-tuning algorithms employed are now called ‘deep learning’. That term is certainly justified in terms of the layers of mathematics being utilized in these deeply nested functions. In deep learning networks, layers upon layers of numerical functions (often Bell Curves, or functions that combine into other, similar, simple shapes) are being moved around fields of numbers, to produce intricately linked mathematical ‘atoms’ of emergent representations.

…

The main thing that brains and connectionist algorithms share in common is opacity, but it would be a huge mistake to think that the algorithms have reached the complexity of brains. The largest known deep learning networks have something like 1010 parameters (the numerical weights in those matrices).19 Most have far fewer, but to get a feel for their maximum size, consider that the length of a year is around 1010 milliseconds. To imagine how much larger and more complex the human brain is than the largest deep learning neural network, imagine stretching each of the milliseconds in that year to be the length of a year themselves. That’s the comparable size of the brain (1020), if we just compare the number of real neural synapses to the number of numbers in the biggest ever deep learning connectionist algorithm.

…

New York: Wiley. 16Ironically, while Babbage’s key innovation of introducing Jacquard loom cards were included in most tabulating machines from the time IBM standardized them in 1928, they only began to be widely used to program general-purpose computers just before Rosenblatt developed the Mark I Perceptron. 17Unfortunately, Turing’s boss, NPL Director Sir Charles Galton Darwin (named for his grandfather Charles Darwin and great-uncle Francis Galton), dismissed the paper as a ‘schoolboy essay’, and it was only published posthumously, in 1968, 14 years after Turing was driven to suicide by the British authorities’ persecution of his crime of homosexuality. 18M. Minsky and S. Papert, 1969, Perceptrons: An Introduction to Computational Geometry. Cambridge, MA: MIT Press. 19Jeremy Hsu, 2015, Biggest Neural Network Ever Pushes AI Deep Learning. IEEE Spectrum, https://spectrum.ieee.org/tech-talk/computing/software/biggest-neural-network-ever-pushes-ai-deep-learning 20Esther M. Sternberg, 2001, The Balance Within: The Science Connecting Health and Emotions. Times Books. 21Antonio Damasio, 2018, The Strange Order of Things: Life, Feeling, and the Making of Cultures. New York: Pantheon. 22Antonio Damasio and Gil Carvalho, 2013, The Nature of Feelings: Evolutionary and Neurobiological Origins.

Futureproof: 9 Rules for Humans in the Age of Automation by Kevin Roose

"World Economic Forum" Davos, adjacent possible, Airbnb, Albert Einstein, algorithmic bias, algorithmic management, Alvin Toffler, Amazon Web Services, Atul Gawande, augmented reality, automated trading system, basic income, Bayesian statistics, Big Tech, big-box store, Black Lives Matter, business process, call centre, choice architecture, coronavirus, COVID-19, data science, deep learning, deepfake, DeepMind, disinformation, Elon Musk, Erik Brynjolfsson, factory automation, fake news, fault tolerance, Frederick Winslow Taylor, Freestyle chess, future of work, Future Shock, Geoffrey Hinton, George Floyd, gig economy, Google Hangouts, GPT-3, hiring and firing, hustle culture, hype cycle, income inequality, industrial robot, Jeff Bezos, job automation, John Markoff, Kevin Roose, knowledge worker, Kodak vs Instagram, labor-force participation, lockdown, Lyft, mandatory minimum, Marc Andreessen, Mark Zuckerberg, meta-analysis, Narrative Science, new economy, Norbert Wiener, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, off-the-grid, OpenAI, pattern recognition, planetary scale, plutocrats, Productivity paradox, QAnon, recommendation engine, remote working, risk tolerance, robotic process automation, scientific management, Second Machine Age, self-driving car, Shoshana Zuboff, Silicon Valley, Silicon Valley startup, social distancing, Steve Jobs, Stuart Kauffman, surveillance capitalism, tech worker, The Future of Employment, The Wealth of Nations by Adam Smith, TikTok, Travis Kalanick, Uber and Lyft, uber lyft, universal basic income, warehouse robotics, Watson beat the top human players on Jeopardy!, work culture

Didn’t their fears always end up being overblown? Several years ago, when I started as a tech columnist for the Times, most of what I heard about AI mirrored my own optimistic views. I met with start-up founders and engineers in Silicon Valley who showed me how new advances in fields like deep learning were helping them build all kinds of world-improving tools: algorithms that could increase farmers’ crop yields, software that would help hospitals run more efficiently, self-driving cars that could shuttle us around while we took naps and watched Netflix. This was the euphoric peak of the AI hype cycle, a time when all of the American tech giants—Google, Facebook, Apple, Amazon, Microsoft—were pouring billions of dollars into developing new AI products and shoving machine learning algorithms into as many of their apps as possible.

…

A 2019 report by Wells Fargo estimated that as many as two hundred thousand finance employees will lose their jobs over the next decade, thanks to tools like these. Medicine is undergoing a machine makeover, as AI learns to do much of the work that used to require trained human specialists. In 2018, a Chinese tech company built a deep learning algorithm that diagnosed brain cancer and other diseases faster and more accurately than a team of fifteen top doctors. The same year, American researchers developed an algorithm capable of identifying malignant tumors on a CT scan with an error rate twenty times lower than a human radiologist.

…

In fact, several companies are already using AI to generate fashion designs. In 2017, an Amazon research team developed a machine learning algorithm that analyzes images of garments in a particular style and learns to generate new garments in that style. Glitch, an AI fashion company started by two MIT graduates, sells pieces that are entirely designed by deep learning algorithms. Will AI spare all TSA agents, or replace all fashion designers? Of course not. But the fallout from automation probably won’t be as tidy as watching some occupations go extinct while others survive without a scratch. * * * — In short, what I should have told the executives at the fancy dinner was that they were asking the wrong question.

The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World by Pedro Domingos

Albert Einstein, Amazon Mechanical Turk, Arthur Eddington, backpropagation, basic income, Bayesian statistics, Benoit Mandelbrot, bioinformatics, Black Swan, Brownian motion, cellular automata, Charles Babbage, Claude Shannon: information theory, combinatorial explosion, computer vision, constrained optimization, correlation does not imply causation, creative destruction, crowdsourcing, Danny Hillis, data is not the new oil, data is the new oil, data science, deep learning, DeepMind, double helix, Douglas Hofstadter, driverless car, Erik Brynjolfsson, experimental subject, Filter Bubble, future of work, Geoffrey Hinton, global village, Google Glasses, Gödel, Escher, Bach, Hans Moravec, incognito mode, information retrieval, Jeff Hawkins, job automation, John Markoff, John Snow's cholera map, John von Neumann, Joseph Schumpeter, Kevin Kelly, large language model, lone genius, machine translation, mandelbrot fractal, Mark Zuckerberg, Moneyball by Michael Lewis explains big data, Narrative Science, Nate Silver, natural language processing, Netflix Prize, Network effects, Nick Bostrom, NP-complete, off grid, P = NP, PageRank, pattern recognition, phenotype, planetary scale, power law, pre–internet, random walk, Ray Kurzweil, recommendation engine, Richard Feynman, scientific worldview, Second Machine Age, self-driving car, Silicon Valley, social intelligence, speech recognition, Stanford marshmallow experiment, statistical model, Stephen Hawking, Steven Levy, Steven Pinker, superintelligent machines, the long tail, the scientific method, The Signal and the Noise by Nate Silver, theory of mind, Thomas Bayes, transaction costs, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, white flight, yottabyte, zero-sum game

Learning Deep Architectures for AI,* by Yoshua Bengio (Now, 2009), is a brief introduction to deep learning. The problem of error signal diffusion in backprop is described in “Learning long-term dependencies with gradient descent is difficult,”* by Yoshua Bengio, Patrice Simard, and Paolo Frasconi (IEEE Transactions on Neural Networks, 1994). “How many computers to identify a cat? 16,000,” by John Markoff (New York Times, 2012), reports on the Google Brain project and its results. Convolutional neural networks, the current deep learning champion, are described in “Gradient-based learning applied to document recognition,”* by Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner (Proceedings of the IEEE, 1998).

…

Hinton, a psychologist turned computer scientist and great-great-grandson of George Boole, the inventor of the logical calculus used in all digital computers, is the world’s leading connectionist. He has tried longer and harder to understand how the brain works than anyone else. He tells of coming home from work one day in a state of great excitement, exclaiming “I did it! I’ve figured out how the brain works!” His daughter replied, “Oh, Dad, not again!” Hinton’s latest passion is deep learning, which we’ll meet later in this chapter. He was also involved in the development of backpropagation, an even better algorithm than Boltzmann machines for solving the credit-assignment problem that we’ll look at next. Boltzmann machines could solve the credit-assignment problem in principle, but in practice learning was very slow and painful, making this approach impractical for most applications.

…

A hard core of connectionists soldiered on, but by and large the attention of the machine-learning field moved elsewhere. (We’ll survey those lands in Chapters 6 and 7.) Today, however, connectionism is resurgent. We’re learning deeper networks than ever before, and they’re setting new standards in vision, speech recognition, drug discovery, and other areas. The new field of deep learning is on the front page of the New York Times. Look under the hood, and . . . surprise: it’s the trusty old backprop engine, still humming. What changed? Nothing much, say the critics: just faster computers and bigger data. To which Hinton and others reply: exactly, we were right all along! In truth, connectionists have made genuine progress.

WTF?: What's the Future and Why It's Up to Us by Tim O'Reilly

"Friedman doctrine" OR "shareholder theory", 4chan, Affordable Care Act / Obamacare, Airbnb, AlphaGo, Alvin Roth, Amazon Mechanical Turk, Amazon Robotics, Amazon Web Services, AOL-Time Warner, artificial general intelligence, augmented reality, autonomous vehicles, barriers to entry, basic income, behavioural economics, benefit corporation, Bernie Madoff, Bernie Sanders, Bill Joy: nanobots, bitcoin, Blitzscaling, blockchain, book value, Bretton Woods, Brewster Kahle, British Empire, business process, call centre, Capital in the Twenty-First Century by Thomas Piketty, Captain Sullenberger Hudson, carbon tax, Carl Icahn, Chuck Templeton: OpenTable:, Clayton Christensen, clean water, cloud computing, cognitive dissonance, collateralized debt obligation, commoditize, computer vision, congestion pricing, corporate governance, corporate raider, creative destruction, CRISPR, crowdsourcing, Danny Hillis, data acquisition, data science, deep learning, DeepMind, Demis Hassabis, Dennis Ritchie, deskilling, DevOps, Didi Chuxing, digital capitalism, disinformation, do well by doing good, Donald Davies, Donald Trump, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, fake news, Filter Bubble, Firefox, Flash crash, Free Software Foundation, fulfillment center, full employment, future of work, George Akerlof, gig economy, glass ceiling, Glass-Steagall Act, Goodhart's law, Google Glasses, Gordon Gekko, gravity well, greed is good, Greyball, Guido van Rossum, High speed trading, hiring and firing, Home mortgage interest deduction, Hyperloop, income inequality, independent contractor, index fund, informal economy, information asymmetry, Internet Archive, Internet of things, invention of movable type, invisible hand, iterative process, Jaron Lanier, Jeff Bezos, jitney, job automation, job satisfaction, John Bogle, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John Zimmer (Lyft cofounder), Kaizen: continuous improvement, Ken Thompson, Kevin Kelly, Khan Academy, Kickstarter, Kim Stanley Robinson, knowledge worker, Kodak vs Instagram, Lao Tzu, Larry Ellison, Larry Wall, Lean Startup, Leonard Kleinrock, Lyft, machine readable, machine translation, Marc Andreessen, Mark Zuckerberg, market fundamentalism, Marshall McLuhan, McMansion, microbiome, microservices, minimum viable product, mortgage tax deduction, move fast and break things, Network effects, new economy, Nicholas Carr, Nick Bostrom, obamacare, Oculus Rift, OpenAI, OSI model, Overton Window, packet switching, PageRank, pattern recognition, Paul Buchheit, peer-to-peer, peer-to-peer model, Ponzi scheme, post-truth, race to the bottom, Ralph Nader, randomized controlled trial, RFC: Request For Comment, Richard Feynman, Richard Stallman, ride hailing / ride sharing, Robert Gordon, Robert Metcalfe, Ronald Coase, Rutger Bregman, Salesforce, Sam Altman, school choice, Second Machine Age, secular stagnation, self-driving car, SETI@home, shareholder value, Silicon Valley, Silicon Valley startup, skunkworks, Skype, smart contracts, Snapchat, Social Responsibility of Business Is to Increase Its Profits, social web, software as a service, software patent, spectrum auction, speech recognition, Stephen Hawking, Steve Ballmer, Steve Jobs, Steven Levy, Stewart Brand, stock buybacks, strong AI, synthetic biology, TaskRabbit, telepresence, the built environment, the Cathedral and the Bazaar, The future is already here, The Future of Employment, the map is not the territory, The Nature of the Firm, The Rise and Fall of American Growth, The Wealth of Nations by Adam Smith, Thomas Davenport, Tony Fadell, Tragedy of the Commons, transaction costs, transcontinental railway, transportation-network company, Travis Kalanick, trickle-down economics, two-pizza team, Uber and Lyft, Uber for X, uber lyft, ubercab, universal basic income, US Airways Flight 1549, VA Linux, warehouse automation, warehouse robotics, Watson beat the top human players on Jeopardy!, We are the 99%, web application, Whole Earth Catalog, winner-take-all economy, women in the workforce, Y Combinator, yellow journalism, zero-sum game, Zipcar

According to Google, RankBrain’s opinion has become the third most important among the more than two hundred factors that it uses to rank pages. Google has also applied deep learning to language translation. The results were so startlingly better that after a few months of testing, the team stopped all work on the old Google Translate system discussed earlier in this chapter and replaced it entirely with the new one based on deep learning. It isn’t yet quite as good as human translators, but it’s close, at least for everyday functional use, though perhaps not for literary purposes. Deep learning is also used in Google Photos. If you have tried Google Photos, you’ve seen how it can recognize objects in your photos.

…

See “We shape our tools and thereafter our tools shape us,” McLuhan Galaxy, April 1, 2013, https://mcluhan galaxy.wordpress.com/2013/04/01/we-shape-our-tools-and-thereafter-our-tools-shape-us/. 165 “what a typical Deep Learning system is”: Lee Gomes, “Facebook AI Director Yann LeCun on His Quest to Unleash Deep Learning and Make Machines Smarter,” IEEE Spectrum, February 28, 2015, http://spectrum.ieee.org/automaton/robotics/artificial-intelligence/face book-ai-director-yann-lecun-on-deep-learning. 165 “can’t run it faster than real time”: Yann LeCun, Facebook post, December 5, 2016, retrieved March 31, 2017, https://m. facebook.com/story.php?story_fbid =10154017359117143&id=722677142. 165 third most important: Sullivan, “FAQ: All About the Google RankBrain Algorithm.” 165 stopped all work on the old Google Translate system: Gideon Lewis-Kraus, “The Great A.I.

…

The trick is to figure out in which direction to tweak each knob and by how much without actually fiddling with them. This involves computing a “gradient,” which for each knob indicates how the light changes when the knob is tweaked. Now, imagine a box with 500 million knobs, 1,000 light bulbs, and 10 million images to train it with. That’s what a typical Deep Learning system is. Deep learning uses layers of recognizers. Before you can recognize a dog, you have to be able to recognize shapes. Before you can recognize shapes, you have to be able to recognize edges, so that you can distinguish a shape from its background. These successive stages of recognition each produce a compressed mathematical representation that is passed up to the next layer.

The Ethical Algorithm: The Science of Socially Aware Algorithm Design by Michael Kearns, Aaron Roth

23andMe, affirmative action, algorithmic bias, algorithmic trading, Alignment Problem, Alvin Roth, backpropagation, Bayesian statistics, bitcoin, cloud computing, computer vision, crowdsourcing, data science, deep learning, DeepMind, Dr. Strangelove, Edward Snowden, Elon Musk, fake news, Filter Bubble, general-purpose programming language, Geoffrey Hinton, Google Chrome, ImageNet competition, Lyft, medical residency, Nash equilibrium, Netflix Prize, p-value, Pareto efficiency, performance metric, personalized medicine, pre–internet, profit motive, quantitative trading / quantitative finance, RAND corporation, recommendation engine, replication crisis, ride hailing / ride sharing, Robert Bork, Ronald Coase, self-driving car, short selling, sorting algorithm, sparse data, speech recognition, statistical model, Stephen Hawking, superintelligent machines, TED Talk, telemarketer, Turing machine, two-sided market, Vilfredo Pareto

The technical name for the algorithmic framework we have been describing is a generative adversarial network (GAN), and the approach we’ve outlined above indeed seems to be highly effective: GANs are an important component of the collection of techniques known as deep learning, which has resulted in qualitative improvements in machine learning for image classification, speech recognition, automatic natural language translation, and many other fundamental problems. (The Turing Award, widely considered the Nobel Prize of computer science, was recently awarded to Yoshua Bengio, Geoffrey Hinton, and Yann LeCun for their pioneering contributions to deep learning.) Fig. 21. Synthetic cat images created by a generative adversarial network (GAN), from https://ajolicoeur.wordpress.com/cats.

…

To see a particularly egregious example, let’s go back to 2015, when the market for machine learning talent was heating up. The techniques of deep learning had recently reemerged from relative obscurity (its previous incarnation was called backpropagation in neural networks, which we discussed in the introduction), delivering impressive results in computer vision and image recognition. But there weren’t yet very many experts who were good at training these algorithms—which was still more of a black art, or perhaps an artisanal craft, than a science. The result was that deep learning experts were commanding salaries and signing bonuses once reserved for Wall Street.

…

But money alone wasn’t enough to recruit talent—top researchers want to work where other top researchers are—so it was important for AI labs that wanted to recruit premium talent to be viewed as places that were already on the cutting edge. In the United States, this included research labs at companies such as Google and Facebook. One way to do this was to beat the big players in a high-profile competition. The ImageNet competition was perfect—focused on exactly the kind of vision task for which deep learning was making headlines. The contest required each team’s computer program to classify the objects in images into a thousand different and highly specific categories, including “frilled lizard,” “banded gecko,” “oscilloscope,” and “reflex camera.” Each team could train their algorithm on a set of 1.5 million images that the competition organizers made available to all participants.

Only Humans Need Apply: Winners and Losers in the Age of Smart Machines by Thomas H. Davenport, Julia Kirby

"World Economic Forum" Davos, AI winter, Amazon Robotics, Andy Kessler, Apollo Guidance Computer, artificial general intelligence, asset allocation, Automated Insights, autonomous vehicles, basic income, Baxter: Rethink Robotics, behavioural economics, business intelligence, business process, call centre, carbon-based life, Clayton Christensen, clockwork universe, commoditize, conceptual framework, content marketing, dark matter, data science, David Brooks, deep learning, deliberate practice, deskilling, digital map, disruptive innovation, Douglas Engelbart, driverless car, Edward Lloyd's coffeehouse, Elon Musk, Erik Brynjolfsson, estate planning, financial engineering, fixed income, flying shuttle, follow your passion, Frank Levy and Richard Murnane: The New Division of Labor, Freestyle chess, game design, general-purpose programming language, global pandemic, Google Glasses, Hans Lippershey, haute cuisine, income inequality, independent contractor, index fund, industrial robot, information retrieval, intermodal, Internet of things, inventory management, Isaac Newton, job automation, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, Joi Ito, Khan Academy, Kiva Systems, knowledge worker, labor-force participation, lifelogging, longitudinal study, loss aversion, machine translation, Mark Zuckerberg, Narrative Science, natural language processing, Nick Bostrom, Norbert Wiener, nuclear winter, off-the-grid, pattern recognition, performance metric, Peter Thiel, precariat, quantitative trading / quantitative finance, Ray Kurzweil, Richard Feynman, risk tolerance, Robert Shiller, robo advisor, robotic process automation, Rodney Brooks, Second Machine Age, self-driving car, Silicon Valley, six sigma, Skype, social intelligence, speech recognition, spinning jenny, statistical model, Stephen Hawking, Steve Jobs, Steve Wozniak, strong AI, superintelligent machines, supply-chain management, tacit knowledge, tech worker, TED Talk, the long tail, transaction costs, Tyler Cowen, Tyler Cowen: Great Stagnation, Watson beat the top human players on Jeopardy!, Works Progress Administration, Zipcar

Determining the meaning and significance of these has always been the province of human beings—and a key aspect of human cognition. But now a wide variety of tools are capable of it, too. Words are increasingly “understood”—counted, classified, interpreted, predicted, etc.—through technologies such as machine learning, natural language processing, neural networks, deep learning, and so forth. Some of the same technologies are being used to analyze and identify images. Humans are still better able to make subjective judgments on unstructured data, such as interpreting the meaning of a poem, or distinguishing between images of good neighborhoods and bad ones. But computers are making headway even on these fronts.

…

Machine learning and neural network analysis is the most promising technology for this application. One branch of machine learning, for example, is particularly well suited to analyzing data in multiple dimensions. Images and video are an example of this type of data—any individual pixel has x and y coordinates, color, intensity, and in videos, time. “Deep learning” neural network approaches have been developed to deal with data in multiple dimensions; the “deep” refers not to “profound,” but rather to a hierarchy of dimensions in the data. It’s this technology that is letting Google engineers identify photos of cats on the Internet. Although it’s difficult to imagine more important tasks than that, perhaps in the near future it will let smart machines watch video taken by drones and security cameras and determine whether something bad is happening.

…

There are already, as we have discussed, systems to understand text and speech, systems to engage in intelligent Q&A with humans, and systems to recognize a variety of images. It’s just that they’re not yet embedded in the brain of a robot. Jim Lawton, the head of products at Rethink Robotics, commented to us in an interview: “An important area of experimentation today is around the intersection of collaborative robots, big data, and deep learning. The goal is to combine automation of physical tasks and cognitive tasks. For example, a robot could start combining all the information about how much torque is applied in a screw. Robots are, after all, a big bucket of sensors. A truly intelligent robot could begin to see what works in terms of how much torque in a screw leads to field failures.

Exponential: How Accelerating Technology Is Leaving Us Behind and What to Do About It by Azeem Azhar

"Friedman doctrine" OR "shareholder theory", "World Economic Forum" Davos, 23andMe, 3D printing, A Declaration of the Independence of Cyberspace, Ada Lovelace, additive manufacturing, air traffic controllers' union, Airbnb, algorithmic management, algorithmic trading, Amazon Mechanical Turk, autonomous vehicles, basic income, Berlin Wall, Bernie Sanders, Big Tech, Bletchley Park, Blitzscaling, Boeing 737 MAX, book value, Boris Johnson, Bretton Woods, carbon footprint, Chris Urmson, Citizen Lab, Clayton Christensen, cloud computing, collective bargaining, computer age, computer vision, contact tracing, contact tracing app, coronavirus, COVID-19, creative destruction, crowdsourcing, cryptocurrency, cuban missile crisis, Daniel Kahneman / Amos Tversky, data science, David Graeber, David Ricardo: comparative advantage, decarbonisation, deep learning, deglobalization, deindustrialization, dematerialisation, Demis Hassabis, Diane Coyle, digital map, digital rights, disinformation, Dissolution of the Soviet Union, Donald Trump, Double Irish / Dutch Sandwich, drone strike, Elon Musk, emotional labour, energy security, Fairchild Semiconductor, fake news, Fall of the Berlin Wall, Firefox, Frederick Winslow Taylor, fulfillment center, future of work, Garrett Hardin, gender pay gap, general purpose technology, Geoffrey Hinton, gig economy, global macro, global pandemic, global supply chain, global value chain, global village, GPT-3, Hans Moravec, happiness index / gross national happiness, hiring and firing, hockey-stick growth, ImageNet competition, income inequality, independent contractor, industrial robot, intangible asset, Jane Jacobs, Jeff Bezos, job automation, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John Perry Barlow, Just-in-time delivery, Kickstarter, Kiva Systems, knowledge worker, Kodak vs Instagram, Law of Accelerating Returns, lockdown, low skilled workers, lump of labour, Lyft, manufacturing employment, Marc Benioff, Mark Zuckerberg, megacity, Mitch Kapor, Mustafa Suleyman, Network effects, new economy, NSO Group, Ocado, offshore financial centre, OpenAI, PalmPilot, Panopticon Jeremy Bentham, Peter Thiel, Planet Labs, price anchoring, RAND corporation, ransomware, Ray Kurzweil, remote working, RFC: Request For Comment, Richard Florida, ride hailing / ride sharing, Robert Bork, Ronald Coase, Ronald Reagan, Salesforce, Sam Altman, scientific management, Second Machine Age, self-driving car, Shoshana Zuboff, Silicon Valley, Social Responsibility of Business Is to Increase Its Profits, software as a service, Steve Ballmer, Steve Jobs, Stuxnet, subscription business, synthetic biology, tacit knowledge, TaskRabbit, tech worker, The Death and Life of Great American Cities, The Future of Employment, The Nature of the Firm, Thomas Malthus, TikTok, Tragedy of the Commons, Turing machine, Uber and Lyft, Uber for X, uber lyft, universal basic income, uranium enrichment, vertical integration, warehouse automation, winner-take-all economy, workplace surveillance , Yom Kippur War

AlexNet had a success rate as high as 87 per cent. Deep learning worked. The triumph of deep learning sparked an AI feeding frenzy. Scientists rushed to build artificial intelligence systems, applying deep neural networks and their derivatives to a vast array of problems: from spotting manufacturing defects to translating between languages; from voice recognition to detecting credit card fraud; from discovering new medicines to recommending the next video we should watch. Investors opened their pocketbooks eagerly to back these inventors. In short order, deep learning was everywhere. As a result, neural networks demanded increasing amounts of data and processing power.

…

By 2010, Moore’s Law had resulted in enough power to facilitate a new kind of machine learning, ‘deep learning’, which involved creating layers of artificial neurons modelled on the cells that underpin human brains. These ‘neural networks’ had long been heralded as the next big thing in AI. Yet they had been stymied by a lack of computational power. Not any more, however. In 2012, a group of leading AI researchers – Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton – developed a ‘deep convolutional neural network’ which applied deep learning to the kinds of image-sorting tasks that AIs had long struggled with. It was rooted in extraordinary computing clout.

…

This giant crane-block battery depends on a particular mix of four, very well-understood technologies – the cranes, the building aggregate, the generator which converts the dropping of blocks to energy, and the shipping systems that let us move these things around. And then there’s a fifth, more unexpected technology: an automated ‘machine vision system’ using deep learning. Each crane has a set of cameras whose input is automatically processed by a computer. This computer controls the cranes and the lifting and placing of the blocks. It obviates any human operators – and it is the absence of a human operator that allows Energy Vault to hit a competitive price. Of course, technologies have always combined.

The Patient Will See You Now: The Future of Medicine Is in Your Hands by Eric Topol

23andMe, 3D printing, Affordable Care Act / Obamacare, Anne Wojcicki, Atul Gawande, augmented reality, Big Tech, bioinformatics, call centre, Clayton Christensen, clean water, cloud computing, commoditize, computer vision, conceptual framework, connected car, correlation does not imply causation, creative destruction, crowdsourcing, dark matter, data acquisition, data science, deep learning, digital divide, disintermediation, disruptive innovation, don't be evil, driverless car, Edward Snowden, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Firefox, gamification, global village, Google Glasses, Google X / Alphabet X, Ignaz Semmelweis: hand washing, information asymmetry, interchangeable parts, Internet of things, Isaac Newton, it's over 9,000, job automation, Julian Assange, Kevin Kelly, license plate recognition, lifelogging, Lyft, Mark Zuckerberg, Marshall McLuhan, meta-analysis, microbiome, Nate Silver, natural language processing, Network effects, Nicholas Carr, obamacare, pattern recognition, personalized medicine, phenotype, placebo effect, quantum cryptography, RAND corporation, randomized controlled trial, Salesforce, Second Machine Age, self-driving car, Silicon Valley, Skype, smart cities, Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia, Snapchat, social graph, speech recognition, stealth mode startup, Steve Jobs, synthetic biology, the scientific method, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, traumatic brain injury, Turing test, Uber for X, uber lyft, Watson beat the top human players on Jeopardy!, WikiLeaks, X Prize

Underlying its predictive capabilities was quite a portfolio of machine learning systems, including Bayesian nets, Markov chains, support vector machine algorithms, and genetic algorithms.33 I won’t go into any more depth; my brain is not smart enough to understand it all, and fortunately it’s not particularly relevant to where we are going here. Another subtype of AI and machine learning,2,20,34–48 known as deep learning, has deep importance to medicine. Deep learning is behind Siri’s ability to decode speech as well as Google Brain experiments to recognize images. Researchers at Google X extracted ten million still images from YouTube videos and fed them to the network of one thousand computers to see what the Brain, with its one million simulated neurons and one billion simulated synapses would come up with on its own.35,36 The answer—cats.

…

M. van Rijmenam, “How Machine Learning Could Result in Great Applications for Your Business,” Big Data-Startups Blog, January 10, 2014, http://www.bigdata-startups.com/machine-learning-result-great-applications-business/. 35. N. Jones, “The Learning Machines,” Nature 505 (2014): 146–148. 36. J. Markoff, “Scientists See Promise in Deep-Learning Programs,” New York Times, November 24, 2012, http://www.nytimes.com/2012/11/24/science/scientists-see-advances-in-deep-learning-a-part-of-artificial-intelligence.html. 37. “Don’t Be Evil, Genius,” The Economist, February 1, 2014, http://www.economist.com/node/21595462/print. 38. J. Pearson, “Superintelligent AI Could Wipe Out Humanity, If We’re Not Ready for It,” Motherboard, April 23, 2014, http://motherboard.vice.com/read/super-intelligent-ai-could-wipe-out-humanity-if-were-not-ready-for-it. 39.

…

McAfee, “The Dawn of the Age of Artificial Intelligence,” The Atlantic, February 2014, http://www.theatlantic.com/business/print/2014/02/the-dawn-of-the-age-of-artificial-intelligence/283730/. 40. S. Schneider, “The Philosophy of ‘Her,’” New York Times, March 2, 2014, http://opinionator.blogs.nytimes.com/2014/03/02/the-philosophy-of-her/?ref=opinion. 41. A. Vance, “The Race to Buy the Human Brains Behind Deep Learning Machines,” Bloomberg Businessweek, January 27, 2014, http://www.businessweek.com/printer/articles/180155-the-race-to-buy-the-human-brains-behind-deep-learning-machines. 42. G. Satell, “Why the Future of Technology Is All Too Human,” Forbes, February 23, 2014, http://www.forbes.com/sites/gregsatell/2014/02/23/why-the-future-of-technology-is-all-too-human/. 43. D. Auerbach, “A.I.

Virtual Competition by Ariel Ezrachi, Maurice E. Stucke

"World Economic Forum" Davos, Airbnb, Alan Greenspan, Albert Einstein, algorithmic management, algorithmic trading, Arthur D. Levinson, barriers to entry, behavioural economics, cloud computing, collaborative economy, commoditize, confounding variable, corporate governance, crony capitalism, crowdsourcing, Daniel Kahneman / Amos Tversky, David Graeber, deep learning, demand response, Didi Chuxing, digital capitalism, disintermediation, disruptive innovation, double helix, Downton Abbey, driverless car, electricity market, Erik Brynjolfsson, Evgeny Morozov, experimental economics, Firefox, framing effect, Google Chrome, independent contractor, index arbitrage, information asymmetry, interest rate derivative, Internet of things, invisible hand, Jean Tirole, John Markoff, Joseph Schumpeter, Kenneth Arrow, light touch regulation, linked data, loss aversion, Lyft, Mark Zuckerberg, market clearing, market friction, Milgram experiment, multi-sided market, natural language processing, Network effects, new economy, nowcasting, offshore financial centre, pattern recognition, power law, prediction markets, price discrimination, price elasticity of demand, price stability, profit maximization, profit motive, race to the bottom, rent-seeking, Richard Thaler, ride hailing / ride sharing, road to serfdom, Robert Bork, Ronald Reagan, search costs, self-driving car, sharing economy, Silicon Valley, Skype, smart cities, smart meter, Snapchat, social graph, Steve Jobs, sunk-cost fallacy, supply-chain management, telemarketer, The Chicago School, The Myth of the Rational Market, The Wealth of Nations by Adam Smith, too big to fail, transaction costs, Travis Kalanick, turn-by-turn navigation, two-sided market, Uber and Lyft, Uber for X, uber lyft, vertical integration, Watson beat the top human players on Jeopardy!, women in the workforce, yield management

Antonio Regalado, “Is Google Cornering the Market on Deep Learning?” MIT Technology Review, January 29, 2014, http://www.technologyreview.com /news/524026/is-google-cornering-the-market-on-deep-learning/; Nicola Jones, “Computer Science: The Learning Machines,” Nature, January 8, 2014, http://www.nature.com/news/computer-science-the-learning-machines -1.14481. European Data Protection Supervisor, Towards a New Digital Ethics: Data, Dignity and Technology, Opinion 4/2015 (September 11, 2015), 9. Robert D. Hof, “Deep Learning,” MIT Technology Review, April 23, 2013, http://www.technologyreview.com/featuredstory/513696/deep-learning/. Tereza Pultarova, “Jaguar Land Rover to Lead Driverless Car Research,” E&T (October 9, 2015), http://eandt.theiet.org /news/2015/oct /jaguar-land -rover-driverless-cars.cfm; David Talbot, “CES 2015: Nvidia Demos a Car Computer Trained with ‘Deep Learning,’ ” MIT Technology Review, January 6, 2015), http://www.technologyreview.com /news/533936/ces-2015 -nvidia-demos-a-car-computer-trained-with-deep-learning /; David Levitin, 2015.

…

Recent years have witnessed groundbreaking research and progress in the design and development of smart, self-learning algorithms to assist in pricing decisions, planning, trade, and logistics. The field has attracted significant investment in deep learning by leading market players.38 In 2011, International Business Machines Corp.’s Jeopardy!-winning Watson computer showcased the power of its deep-learning techniques, which enabled the computer to optimize its strategy following trials and feedback.39 Since then, IBM has invested in widening the capacity and functionality of the technology, with the aim of making it “the equivalent of a computing operating system for an emerging class of data-fueled artificialintelligence applications.”40 Recently, the launch of the Deep Q network by Google showcased enhanced self-learning capacity.

…

Tereza Pultarova, “Jaguar Land Rover to Lead Driverless Car Research,” E&T (October 9, 2015), http://eandt.theiet.org /news/2015/oct /jaguar-land -rover-driverless-cars.cfm; David Talbot, “CES 2015: Nvidia Demos a Car Computer Trained with ‘Deep Learning,’ ” MIT Technology Review, January 6, 2015), http://www.technologyreview.com /news/533936/ces-2015 -nvidia-demos-a-car-computer-trained-with-deep-learning /; David Levitin, 2015. “The Sum of Human Knowledge,” Wall Street Journal, September 18, 2015, http://www.wsj.com /articles/the-sum-of-human -knowledge-1442610803. Lohr, “IBM’s AI System Watson to Get Second Home.” European Data Protection Supervisor, Towards a New Digital Ethics.

The Inevitable: Understanding the 12 Technological Forces That Will Shape Our Future by Kevin Kelly

A Declaration of the Independence of Cyberspace, Aaron Swartz, AI winter, Airbnb, Albert Einstein, Alvin Toffler, Amazon Web Services, augmented reality, bank run, barriers to entry, Baxter: Rethink Robotics, bitcoin, blockchain, book scanning, Brewster Kahle, Burning Man, cloud computing, commoditize, computer age, Computer Lib, connected car, crowdsourcing, dark matter, data science, deep learning, DeepMind, dematerialisation, Downton Abbey, driverless car, Edward Snowden, Elon Musk, Filter Bubble, Freestyle chess, Gabriella Coleman, game design, Geoffrey Hinton, Google Glasses, hive mind, Howard Rheingold, index card, indoor plumbing, industrial robot, Internet Archive, Internet of things, invention of movable type, invisible hand, Jaron Lanier, Jeff Bezos, job automation, John Markoff, John Perry Barlow, Kevin Kelly, Kickstarter, lifelogging, linked data, Lyft, M-Pesa, machine readable, machine translation, Marc Andreessen, Marshall McLuhan, Mary Meeker, means of production, megacity, Minecraft, Mitch Kapor, multi-sided market, natural language processing, Netflix Prize, Network effects, new economy, Nicholas Carr, off-the-grid, old-boy network, peer-to-peer, peer-to-peer lending, personalized medicine, placebo effect, planetary scale, postindustrial economy, Project Xanadu, recommendation engine, RFID, ride hailing / ride sharing, robo advisor, Rodney Brooks, self-driving car, sharing economy, Silicon Valley, slashdot, Snapchat, social graph, social web, software is eating the world, speech recognition, Stephen Hawking, Steven Levy, Ted Nelson, TED Talk, The future is already here, the long tail, the scientific method, transport as a service, two-sided market, Uber for X, uber lyft, value engineering, Watson beat the top human players on Jeopardy!, WeWork, Whole Earth Review, Yochai Benkler, yottabyte, zero-sum game

It can take many millions of these nodes (each one producing a calculation feeding others around it), stacked up to 15 levels high, to recognize a human face. In 2006, Geoff Hinton, then at the University of Toronto, made a key tweak to this method, which he dubbed “deep learning.” He was able to mathematically optimize results from each layer so that the learning accumulated faster as it proceeded up the stack of layers. Deep-learning algorithms accelerated enormously a few years later when they were ported to GPUs. The code of deep learning alone is insufficient to generate complex logical thinking, but it is an essential component of all current AIs, including IBM’s Watson; DeepMind, Google’s search engine; and Facebook’s algorithms.

…

thousand games of chess: Personal correspondence with Daylen Yang (author of the Stockfish chess app), Stefan Meyer-Kahlen (developed the multiple award-winning computer chess program Shredder), and Danny Kopec (American chess International Master and cocreator of one of the standard computer chess testing systems), September 2014. “akin to building a rocket ship”: Caleb Garling, “Andrew Ng: Why ‘Deep Learning’ Is a Mandate for Humans, Not Just Machines,” Wired, May 5, 2015. In 2006, Geoff Hinton: Kate Allen, “How a Toronto Professor’s Research Revolutionized Artificial Intelligence,” Toronto Star, April 17, 2015. he dubbed “deep learning”: Yann LeCun, Yoshua Bengio, and Geoffrey Hinton, “Deep Learning,” Nature 521, no. 7553 (2015): 436–44. the network effect: Carl Shapiro and Hal R. Varian, Information Rules: A Strategic Guide to the Network Economy (Boston: Harvard Business Review Press, 1998).

…

Every time you type a query, click on a search-generated link, or create a link on the web, you are training the Google AI. When you type “Easter Bunny” into the image search bar and then click on the most Easter Bunny–looking image, you are teaching the AI what an Easter Bunny looks like. Each of the 3 billion queries that Google conducts each day tutors the deep-learning AI over and over again. With another 10 years of steady improvements to its AI algorithms, plus a thousandfold more data and a hundred times more computing resources, Google will have an unrivaled AI. In a quarterly earnings conference call in the fall of 2015, Google CEO Sundar Pichai stated that AI was going to be “a core transformative way by which we are rethinking everything we are doing. . . .

Machines of Loving Grace: The Quest for Common Ground Between Humans and Robots by John Markoff

A Declaration of the Independence of Cyberspace, AI winter, airport security, Andy Rubin, Apollo 11, Apple II, artificial general intelligence, Asilomar, augmented reality, autonomous vehicles, backpropagation, basic income, Baxter: Rethink Robotics, Bill Atkinson, Bill Duvall, bioinformatics, Boston Dynamics, Brewster Kahle, Burning Man, call centre, cellular automata, Charles Babbage, Chris Urmson, Claude Shannon: information theory, Clayton Christensen, clean water, cloud computing, cognitive load, collective bargaining, computer age, Computer Lib, computer vision, crowdsourcing, Danny Hillis, DARPA: Urban Challenge, data acquisition, Dean Kamen, deep learning, DeepMind, deskilling, Do you want to sell sugared water for the rest of your life?, don't be evil, Douglas Engelbart, Douglas Engelbart, Douglas Hofstadter, Dr. Strangelove, driverless car, dual-use technology, Dynabook, Edward Snowden, Elon Musk, Erik Brynjolfsson, Evgeny Morozov, factory automation, Fairchild Semiconductor, Fillmore Auditorium, San Francisco, From Mathematics to the Technologies of Life and Death, future of work, Galaxy Zoo, General Magic , Geoffrey Hinton, Google Glasses, Google X / Alphabet X, Grace Hopper, Gunnar Myrdal, Gödel, Escher, Bach, Hacker Ethic, Hans Moravec, haute couture, Herbert Marcuse, hive mind, hype cycle, hypertext link, indoor plumbing, industrial robot, information retrieval, Internet Archive, Internet of things, invention of the wheel, Ivan Sutherland, Jacques de Vaucanson, Jaron Lanier, Jeff Bezos, Jeff Hawkins, job automation, John Conway, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John Perry Barlow, John von Neumann, Kaizen: continuous improvement, Kevin Kelly, Kiva Systems, knowledge worker, Kodak vs Instagram, labor-force participation, loose coupling, Marc Andreessen, Mark Zuckerberg, Marshall McLuhan, medical residency, Menlo Park, military-industrial complex, Mitch Kapor, Mother of all demos, natural language processing, Neil Armstrong, new economy, Norbert Wiener, PageRank, PalmPilot, pattern recognition, Philippa Foot, pre–internet, RAND corporation, Ray Kurzweil, reality distortion field, Recombinant DNA, Richard Stallman, Robert Gordon, Robert Solow, Rodney Brooks, Sand Hill Road, Second Machine Age, self-driving car, semantic web, Seymour Hersh, shareholder value, side project, Silicon Valley, Silicon Valley startup, Singularitarianism, skunkworks, Skype, social software, speech recognition, stealth mode startup, Stephen Hawking, Steve Ballmer, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, Strategic Defense Initiative, strong AI, superintelligent machines, tech worker, technological singularity, Ted Nelson, TED Talk, telemarketer, telepresence, telepresence robot, Tenerife airport disaster, The Coming Technological Singularity, the medium is the message, Thorstein Veblen, Tony Fadell, trolley problem, Turing test, Vannevar Bush, Vernor Vinge, warehouse automation, warehouse robotics, Watson beat the top human players on Jeopardy!, We are as Gods, Whole Earth Catalog, William Shockley: the traitorous eight, zero-sum game

As a consultant Hinton had introduced the deep learning neural net approach early on at Microsoft, and he was vindicated in 2012, when Microsoft’s head of research Richard Rashid gave a lecture in a vast auditorium in Tianjin, China. As the research executive spoke in English he paused after each sentence, which was then immediately translated by software into spoken Chinese in a simulation of his own voice. At the end of the talk, there was silence and then stunned applause from the audience. The demonstration hadn’t been perfect, but by adding deep learning algorithm techniques the company had adopted from Hinton’s research, it had been able to reduce recognition errors by more than 30 percent.

…

Thinking of themselves as the “three musketeers,” Hinton, LeCun, and Bengio set out to change that. Beginning in 2004 they embarked on a “conspiracy”—in LeCun’s words—to boost the popularity of the networks, complete with a rebranding campaign offering more alluring concepts of the technology such as “deep learning” and “deep belief nets.” LeCun had by this time moved to New York University, partly for closer ties with neuroscientists and with researchers applying machine-learning algorithms to the problem of vision. Hinton approached a Canadian foundation, the Canadian Institute for Advanced Research, for support to organize a research effort in the field and to hold several workshops each year.

…

It had a broad portfolio of research projects, stretching from Thrun’s work in autonomous cars to efforts to scale up neural networks, loosely identified as “brain” projects, evoking a new wave of AI. The Human Brain Project was initially led by Andrew Ng, who had been a colleague with Thrun at the resurrected Stanford Artificial Intelligence Laboratory. Ng was an expert in machine learning and adept in some of the deep learning neural network techniques that Hinton and LeCun had pioneered. In 2011, he began spending time at Google building a machine vision system and the following year it had matured to the point where Google researchers presented a paper on how the network performed in an unsupervised learning experiment using YouTube videos.

Text Analytics With Python: A Practical Real-World Approach to Gaining Actionable Insights From Your Data by Dipanjan Sarkar

bioinformatics, business intelligence, business logic, computer vision, continuous integration, data science, deep learning, Dr. Strangelove, en.wikipedia.org, functional programming, general-purpose programming language, Guido van Rossum, information retrieval, Internet of things, invention of the printing press, iterative process, language acquisition, machine readable, machine translation, natural language processing, out of africa, performance metric, premature optimization, recommendation engine, self-driving car, semantic web, sentiment analysis, speech recognition, statistical model, text mining, Turing test, web application

Dipanjan’s interests include learning about new technology, financial markets, disruptive startups, data science, and more recently, artificial intelligence and deep learning. In his spare time he loves reading, gaming, and watching popular sitcoms and football. About the Technical Reviewer Shanky Sharma Currently leading the AI team at Nextremer India, Shanky Sharma’s work entails implementing various AI and machine learning–related projects and working on deep learning for speech recognition in Indic languages. He hopes to grow and scale new horizons in AI and machine learning technologies. Statistics intrigue him and he loves playing with numbers, designing algorithms, and giving solutions to people.

…

Besides these, there are several other frameworks and libraries that are not dedicated towards text analytics but that are useful when you want to use machine learning techniques on textual data. These include the scikit-learn, numpy, and scipy stack. Besides these, deep learning and tensor-based libraries like theano, tensorflow, and keras also come in handy if you want to build advanced deep learning models based on deep neural nets, convnets, and LSTM-based models . You can install most of these libraries using the pip install <library> command from the command prompt or terminal. We will talk about any caveats if present in the upcoming chapters when we use these libraries.

…

Besides being a multipurpose language, the wide variety of frameworks, libraries, and platforms that have been developed by using Python and to be used for Python form a complete robust ecosystem around Python. These libraries make life easier by giving us a wide variety of capabilities and functionality to perform various tasks with minimal code. Some examples would be libraries for handling databases, text data, machine learning, signal processing, image processing, deep learning, artificial intelligence—and the list goes on. Open source: As open source, Python is actively developed and updated constantly with improvements, optimizations, and new features. Now the Python Software Foundation (PSF) owns all Python-related intellectual property (IP) and administers all license-related issues.

The Dawn of Eurasia: On the Trail of the New World Order by Bruno Macaes

active measures, Berlin Wall, Brexit referendum, British Empire, computer vision, deep learning, Deng Xiaoping, different worldview, digital map, Donald Trump, energy security, European colonialism, eurozone crisis, failed state, Francis Fukuyama: the end of history, gentrification, geopolitical risk, global value chain, illegal immigration, intermodal, iterative process, land reform, liberal world order, Malacca Straits, mass immigration, megacity, middle-income trap, open borders, Parag Khanna, savings glut, scientific worldview, Silicon Valley, South China Sea, speech recognition, Suez canal 1869, The Brussels Effect, trade liberalization, trade route, Transnistria, young professional, zero-sum game, éminence grise

The comparison forces itself upon the visitor to Baidu, who is greeted in the garden and then the lobby by massive slides connecting to the upper levels, the symbol of fast-paced internet companies worldwide. The Institute of Deep Learning is one of the ways that Baidu, the Chinese search giant, is trying to keep ahead of the innovation pack, notably by making sure that the latest technological advances can quickly be used across its different businesses, including the core search algorithm. Deep learning is an old idea in artificial intelligence and many think it is our best hope in building software that will get us very near to – and in some cases surpass – human abilities.

…

In this, machine intelligence comes to resemble the way a large array of neurons works in the human brain. Speech and image recognition are among the most immediate applications of deep learning. Yuanqing told me how Baidu had been able to develop practically infallible speech recognition applications, even if the user chooses to whisper to his device rather than speak. They were now concentrating their efforts on how to apply deep learning to automated driving. Applying it to prediction systems still lies considerably in the future, but the future is getting closer each day. ‘How would you describe what is different about the way the Chinese approach technology?’

…

‘It learns to recognize speech and the learning process works equally for every language. Feed it the data and it will learn Latin or Sanskrit. Some algorithms have even invented their own languages.’ I had come to the Baidu Technology Park in the Haidian District of Beijing to meet Yuanqing Lin, Director of the Baidu Institute of Deep Learning. The complex consists of five individual buildings connected by gallery bridges and overlooking a central botanical garden where small streams of water and recently planted trees gradually reveal a space capsule in the centre. It is easy to get lost here. Haidian hosts a number of technology parks, each with tens or hundreds of both established companies and start-ups, a scale perhaps still half of Silicon Valley but fast approaching it.

The Science and Technology of Growing Young: An Insider's Guide to the Breakthroughs That Will Dramatically Extend Our Lifespan . . . And What You Can Do Right Now by Sergey Young

23andMe, 3D printing, Albert Einstein, artificial general intelligence, augmented reality, basic income, Big Tech, bioinformatics, Biosphere 2, brain emulation, caloric restriction, caloric restriction, Charles Lindbergh, classic study, clean water, cloud computing, cognitive bias, computer vision, coronavirus, COVID-19, CRISPR, deep learning, digital twin, diversified portfolio, Doomsday Clock, double helix, Easter island, Elon Musk, en.wikipedia.org, epigenetics, European colonialism, game design, Gavin Belson, George Floyd, global pandemic, hockey-stick growth, impulse control, Internet of things, late capitalism, Law of Accelerating Returns, life extension, lockdown, Lyft, Mark Zuckerberg, meta-analysis, microbiome, microdosing, moral hazard, mouse model, natural language processing, personalized medicine, plant based meat, precision agriculture, radical life extension, Ralph Waldo Emerson, Ray Kurzweil, Richard Feynman, ride hailing / ride sharing, Ronald Reagan, self-driving car, seminal paper, Silicon Valley, stem cell, Steve Jobs, tech billionaire, TED Talk, uber lyft, ultra-processed food, universal basic income, Virgin Galactic, Vision Fund, X Prize

Now this algorithm is hard at work for the British National Health System, proactively analyzing data livestreamed from outpatients. When a potential flare-up is indicated by KenSci’s AI, doctors are alerted so that they can intervene before the situation becomes an emergency. 2.AI Case Study #2: Deep Learning and Computer Vision for Diagnosis Diabetic retinopathy (DR) is a complication of diabetes. Over time, excess sugar damages tiny blood vessels connected to the retina. The body grows new blood vessels, but they rupture easily. Untreated, this eventually leads to total blindness. If caught early, it is highly treatable, but it is fairly asymptomatic at first, and ophthalmologists who can identify the disease are rare.

…

With thirteen million people in his home country of India suffering from DR, Google scientist Varun Gulshan knew there must be a better way to diagnose and treat the disease using AI. His team first obtained one million retinal scans that had already been analyzed and graded by ophthalmologists. They then used the AI techniques of deep learning and computer vision to teach their algorithm to recognize DR, just like a qualified ophthalmologist would. Today, the shortage of doctors to monitor the retinal condition of diabetic patients is less of a problem, thanks to AI. 3.AI Case Study #3: Natural Language Processing and Taking AI Health Care to the Next Level Using AI to analyze raw data and even images is one thing.

…

Medical records can be monitored to identify patterns associated with negative health events like hospital-acquired infections, heart attacks, and so on. On a pretty narrow basis, NLP can already perform the detailed analysis I described above to enhance physician decision making. In time, AI will be able to combine computer vision, deep learning, natural language processing, and other techniques to provide extremely reliable diagnostic outcomes. It will take all of the guesswork and inconsistency out of medical care and make our old one-size-fits-all approach seem barbaric in retrospect. We have a long way to go, but within the Near Horizon of Longevity, precision medicine will become, without a great deal of hyperbole, perfect medicine.

Hello World: Being Human in the Age of Algorithms by Hannah Fry

23andMe, 3D printing, Air France Flight 447, Airbnb, airport security, algorithmic bias, algorithmic management, augmented reality, autonomous vehicles, backpropagation, Brixton riot, Cambridge Analytica, chief data officer, computer vision, crowdsourcing, DARPA: Urban Challenge, data science, deep learning, DeepMind, Douglas Hofstadter, driverless car, Elon Musk, fake news, Firefox, Geoffrey Hinton, Google Chrome, Gödel, Escher, Bach, Ignaz Semmelweis: hand washing, John Markoff, Mark Zuckerberg, meta-analysis, Northpointe / Correctional Offender Management Profiling for Alternative Sanctions, pattern recognition, Peter Thiel, RAND corporation, ransomware, recommendation engine, ride hailing / ride sharing, selection bias, self-driving car, Shai Danziger, Silicon Valley, Silicon Valley startup, Snapchat, sparse data, speech recognition, Stanislav Petrov, statistical model, Stephen Hawking, Steven Levy, systematic bias, TED Talk, Tesla Model S, The Wisdom of Crowds, Thomas Bayes, trolley problem, Watson beat the top human players on Jeopardy!, web of trust, William Langewiesche, you are the product

In that perfect storm of misplaced trust and power and influence, the consequences have the potential to fundamentally alter our society. * This is paraphrased from a comment made by the computer scientist and machine-learning pioneer Andrew Ng in a talk he gave in 2015. See Tech Events, ‘GPU Technology Conference 2015 day 3: What’s Next in Deep Learning’, YouTube, 20 Nov. 2015, https://www.youtube.com/watch?v=qP9TOX8T-kI. † Simulating the brain of a worm is precisely the goal of the international science project OpenWorm. They’re hoping to artificially reproduce the network of 302 neurons found within the brain of the C. elegans worm. To put that into perspective, we humans have around 100,000,000,000 neurons.

…

This is another ‘machine-learning algorithm’, like the random forests we met in the ‘Justice’ chapter. It goes beyond what the operators program it to do and learns itself from the images it’s given. It’s this ability to learn that endows the algorithm with ‘artificial intelligence’. And the many layers of knobs and dials also give the network a deep structure, hence the term ‘deep learning’. Neural networks have been around since the middle of the twentieth century, but until quite recently we’ve lacked the widespread access to really powerful computers necessary to get the best out of them. The world was finally forced to sit up and take them seriously in 2012 when computer scientist Geoffrey Hinton and two of his students entered a new kind of neural network into an image recognition competition.12 The challenge was to recognize – among other things – dogs.

…

The world was finally forced to sit up and take them seriously in 2012 when computer scientist Geoffrey Hinton and two of his students entered a new kind of neural network into an image recognition competition.12 The challenge was to recognize – among other things – dogs. Their artificially intelligent algorithm blew the best of its competitors out of the water and kicked off a massive renaissance in deep learning. An algorithm that works without our knowing how it makes its decisions might sound like witchcraft, but it might not be all that dissimilar from how we learn ourselves. Consider this comparison. One team recently trained an algorithm to distinguish between photos of wolves and pet huskies. They then showed how, thanks to the way it had tuned its own dials, the algorithm wasn’t using anything to do with the dogs as clues at all.

The Hype Machine: How Social Media Disrupts Our Elections, Our Economy, and Our Health--And How We Must Adapt by Sinan Aral

Airbnb, Albert Einstein, algorithmic bias, AlphaGo, Any sufficiently advanced technology is indistinguishable from magic, AOL-Time Warner, augmented reality, behavioural economics, Bernie Sanders, Big Tech, bitcoin, Black Lives Matter, Cambridge Analytica, carbon footprint, Cass Sunstein, computer vision, contact tracing, coronavirus, correlation does not imply causation, COVID-19, crowdsourcing, cryptocurrency, data science, death of newspapers, deep learning, deepfake, digital divide, digital nomad, disinformation, disintermediation, Donald Trump, Drosophila, Edward Snowden, Elon Musk, en.wikipedia.org, end-to-end encryption, Erik Brynjolfsson, experimental subject, facts on the ground, fake news, Filter Bubble, George Floyd, global pandemic, hive mind, illegal immigration, income inequality, Kickstarter, knowledge worker, lockdown, longitudinal study, low skilled workers, Lyft, Mahatma Gandhi, Mark Zuckerberg, Menlo Park, meta-analysis, Metcalfe’s law, mobile money, move fast and break things, multi-sided market, Nate Silver, natural language processing, Neal Stephenson, Network effects, performance metric, phenotype, recommendation engine, Robert Bork, Robert Shiller, Russian election interference, Second Machine Age, seminal paper, sentiment analysis, shareholder value, Sheryl Sandberg, skunkworks, Snapchat, social contagion, social distancing, social graph, social intelligence, social software, social web, statistical model, stem cell, Stephen Hawking, Steve Bannon, Steve Jobs, Steve Jurvetson, surveillance capitalism, Susan Wojcicki, Telecommunications Act of 1996, The Chicago School, the strength of weak ties, The Wisdom of Crowds, theory of mind, TikTok, Tim Cook: Apple, Uber and Lyft, uber lyft, WikiLeaks, work culture , Yogi Berra