Amazon Web Services

143 results back to index


pages: 44 words: 12,675

Turning the Flywheel: A Monograph to Accompany Good to Great by Jim Collins

Amazon Web Services, index fund, Jeff Bezos, Socratic dialogue, Vanguard fund

Most seeming “second flywheels” come about organically, as bullet-to-cannonball extensions of a primary flywheel. Amazon showed this precise pattern with its Amazon Web Services, which enables organizations big and small to efficiently buy computing power, store data, host websites, and avail themselves of other technology services. Amazon Web Services began as an internal system to provide backend technology support for Amazon’s own e-commerce efforts. In 2006, the company fired a bullet on offering these very same services to outside companies. The bullet hit its target and Amazon had enough calibration to fire a cannonball. A decade later, Amazon Web Services (while still contributing less than 10 percent of Amazon’s overall net sales) generated a substantial portion of Amazon’s operating income.21 Even though Amazon Web Services first appears to be a very different activity than the consumer-retail business, it has substantial similarities.

Hansen, Great by Choice: Uncertainty, Chaos and Luck—Why Some Thrive Despite Them All (New York, NY: HarperBusiness, 2011), 91–95. 21Amazon, Fiscal 2015 Annual Letter to Shareholders (Seattle, WA: Amazon, 2015); Amazon, Fiscal 2016 Annual Report (Seattle, WA: Amazon, 2016); Amazon, Fiscal 2017 Annual Report (Seattle, WA: Amazon, 2017); Alex Hern, “Amazon Web Services: the secret to the online retailer’s future success,” The Guardian, February 2, 2017, https://www.theguardian.com/technology/2017/feb/02/amazon-web-services-the-secret-to-the-online-retailers-future-success; Robert Hof, “Ten Years Later, Amazon Web Services Defies Skeptics,” Forbes, March 22, 2016, https://www.forbes.com/sites/roberthof/2016/03/22/ten-years-later-amazon-web-services-defies-skeptics/#244356466c44. 22Amazon, Fiscal 2017 Annual Report (Seattle, WA: Amazon, 2017); “Global Retail Industry Worth USD 28 Trillion by 2019 - Analysis, Technologies & Forecasts Report 2016–2019 - Research and Markets,” Business Wire, June 30, 2016, https://www.businesswire.com/news/home/20160630005551/en/Global-Retail-Industry-Worth-USD-28-Trillion. 23Jim Collins, How the Mighty Fall: And Why Some Companies Never Give In (Boulder, CO: Jim Collins, 2009), 29–36. 24Alan Wurtzel, Good to Great to Gone: The 60 Year Rise and Fall of Circuit City (New York, NY: Diversion Books, Kindle Edition, 2012), Chapter 8; Circuit City Stores, Inc., Fiscal 2002 Annual Report (Richmond, VA: Circuit City Stores, Inc., 2002); Michael Janofsky, “Circuit City Takes a Spin at Used Car Marketing,” The New York Times, October 25, 1993, http://www.nytimes.com/1993/10/25/business/circuit-city-takes-a-spin-at-used-car-marketing.html; Mike McKesson, “Circuit City at Wheel of New Deal for Used-Car Shoppers: Megastores,” Los Angeles Times, January 28, 1996, http://articles.latimes.com/1996–01-28/news/mn-29582_1_circuit-city. 25Alan Wurtzel, Good to Great to Gone: The 60 Year Rise and Fall of Circuit City (New York, NY: Diversion Books, Kindle Edition, 2012), Chapter 8; Circuit City Stores, Inc., Fiscal 2002 Annual Report (Richmond, VA: Circuit City Stores, Inc., 2002). 26Alan Wurtzel, Good to Great to Gone: The 60 Year Rise and Fall of Circuit City (New York, NY: Diversion Books, Kindle Edition, 2012), Loc 3542 of 5094. 27Alan Wurtzel, Good to Great to Gone: The 60 Year Rise and Fall of Circuit City (New York, NY: Diversion Books, Kindle Edition, 2012), Chapter 8 and Chapter 10; Jesse Romero, “The Rise and Fall of Circuit City,” Federal Reserve Bank of Richmond, 2013; Jim Collins, How the Mighty Fall: And Why Some Companies Never Give In (Boulder, CO: Jim Collins, 2009), 29–36.

A decade later, Amazon Web Services (while still contributing less than 10 percent of Amazon’s overall net sales) generated a substantial portion of Amazon’s operating income.21 Even though Amazon Web Services first appears to be a very different activity than the consumer-retail business, it has substantial similarities. As Bezos wrote in his 2015 annual letter to shareholders, “Superficially, the two could hardly be more different. One serves consumers and the other serves enterprises . . . Under the surface, the two are not so different after all.” Amazon Web Services aims to lower prices and expand offerings to an ever-growing cadre of customers, leading to increasing revenues per fixed costs, which then drives the flywheel around again. The whole idea is to make it as easy and cost-effective for enterprises to meet their technology needs as it is for consumers buying personal stuff at the Amazon marketplace. Sure, there are differences in how the two businesses operate, but the two are more like fraternal twins than being from an entirely different family lineage.


Learning Ansible 2 - Second Edition by Fabio Alessandro Locati

Amazon Web Services, anti-pattern, cloud computing, continuous integration, Debian, DevOps, don't repeat yourself, Infrastructure as a Service, inventory management, Kickstarter, revision control, source of truth, web application

In the next sections we will provision the environment that we have used in the previous chapters (two web servers and one database server) in the following environments: • Simple Amazon Web Service deployment: Where all machines will be placed in the same Availability Zone and same network • Complex Amazon Web Service deployment: Where the machines will be split in multiple Availability Zones as well as networks • DigitalOcean: DigitalOcean does not allow us to do many networking tweaks so it will be similar to the first one • Docker: We will create a simple deployment in this case Amazon Web Service Amazon Web Service is the most used public cloud by a fair amount and it's often chosen due to their huge amount of available services as well as the huge amount of documentation, answered questions, and articles that can be expected from such a popular product.

The Ansible inventory scripts can be written in any language but, for consistency reasons, dynamic inventory scripts should be written in Python. Remember that these scripts need to be executable directly, so please remember to set them with the executable flag (chmod + x inventory.py). In this chapter, we will take a look at Amazon Web Services and DigitalOcean scripts that can be downloaded from the official Ansible repository. Amazon Web Services To allow Ansible to gather data from Amazon Web Services (AWS) about your EC2 instances, you need to download the following two files from Ansible's GitHub repository at https://github.com/ ansible/ansible: • The ec2.py inventory script • The ec2.ini file, which contains the configuration for your EC2 inventory script Ansible uses AWS Python library boto to communicate with AWS using APIs.

To simplify this, Amazon provides some widely used DBaaS, more specifically: • • • • • • Aurora MariaDB MySQL Oracle PostgreSQL SQL Server For each one of those engines, Amazon offers different features and price models but the specifics of each is beyond the goal of this book. Setting up an account with AWS The first thing we will need before starting to work on our Amazon Web Service is an account. Creating an account on Amazon Web Services is pretty straightforward and very well-documented by Amazon official documentation as well as by multiple independent sites and therefore it will not be covered in these pages. After you have created your AWS account, you need to go into the AWS and do the following: • Upload your SSH key in EC2 | Keypairs • Create a new user in Identity & Access Management | Users | Create new user and create a file in ~/.aws/credentials with the following lines: [default] aws_access_key_id = YOUR_ACCESS_KEY aws_secret_access_key = YOUR_SECRET_KEY After you have created your AWS Keys and uploaded your SSH key, you need to set up Route53.


pages: 222 words: 54,506

One Click: Jeff Bezos and the Rise of Amazon.com by Richard L. Brandt

Amazon Web Services, automated trading system, big-box store, call centre, cloud computing, Dynabook, Elon Musk, inventory management, Jeff Bezos, Kevin Kelly, Kickstarter, Marc Andreessen, new economy, science of happiness, search inside the book, Silicon Valley, Silicon Valley startup, skunkworks, software patent, Steve Jobs, Stewart Brand, Tony Hsieh, Whole Earth Catalog, Y2K

Netflix can’t afford (at least not yet) to buy all the computing power needed to load up films instantly and stream them to thousands of customers at any moment. So it rents computers from Amazon’s vast store at pennies per minute to handle the tasks, tapping into just as much computer power as it needs at any given moment. It’s all part of a surprising business from the online retailing company, called Amazon Web Services, which is part of a larger trend known as cloud computing. Services like this bring in half a billion dollars annually in revenues to Amazon. Buying companies is a relatively easy way for a stock-rich company to expand its business. But sometimes a great executive will stumble upon an unexpected new idea, or one of his employees may come up with something. The key is the ability to look beyond the current conventional wisdom and embrace a radical new idea.

That prevented sensitive data from leaking out. Despite some concerns from higher-ups about security, Frederick recalled, “The funny thing is that it did not take a great deal of convincing.” The executives soon realized that they could have a gold mine in their cubicles. By making its data and tools available to outside programmers, Amazon could actually outsource the development of new products—for free. Amazon Web Services was launched in July 2002. “We’re putting out a welcome mat for developers,” announced Bezos. “This is an important beginning and new direction for us.” Developers began creating new sites with original features that could send new buyers to Amazon and help them find and buy products. One former Amazon developer, for example, created a site that he dubbed “Amazon Light.” It included a search box to find any product for sale at Amazon and sported the company’s “Buy” button, but added a feature.

What was this, the Macy’s Santa from Miracle on 34th Street? No, but it was a great feature that benefitted customers. Besides, if the customers did decide to buy from the outside site, the transaction actually went through Amazon, which collects a fee for the transaction. Site owners who sent customers to Amazon to buy Amazon products got about a 15 percent cut of any sale they sent to Amazon. Web services—the process of sharing services between Web sites—had been discussed by Internet pundits for years. Amazon was the first to make the concept a reality in a big way. About two years after the launch of Web services, Amazon boasted sixty-five thousand developers using the program and sending some ten million queries a day to Amazon’s servers.That’s a lot of new customers. Further, the offering began to look a lot like the phenomenon now known as cloud computing—tapping into a program sitting on a Web server somewhere rather than one sitting on your own desktop.


pages: 380 words: 118,675

The Everything Store: Jeff Bezos and the Age of Amazon by Brad Stone

airport security, Amazon Mechanical Turk, Amazon Web Services, bank run, Bernie Madoff, big-box store, Black Swan, book scanning, Brewster Kahle, buy and hold, call centre, centre right, Chuck Templeton: OpenTable:, Clayton Christensen, cloud computing, collapse of Lehman Brothers, crowdsourcing, cuban missile crisis, Danny Hillis, Douglas Hofstadter, Elon Musk, facts on the ground, game design, housing crisis, invention of movable type, inventory management, James Dyson, Jeff Bezos, John Markoff, Kevin Kelly, Kodak vs Instagram, late fees, loose coupling, low skilled workers, Maui Hawaii, Menlo Park, Network effects, new economy, optical character recognition, pets.com, Ponzi scheme, quantitative hedge fund, recommendation engine, Renaissance Technologies, RFID, Rodney Brooks, search inside the book, shareholder value, Silicon Valley, Silicon Valley startup, six sigma, skunkworks, Skype, statistical arbitrage, Steve Ballmer, Steve Jobs, Steven Levy, Stewart Brand, Thomas L Friedman, Tony Hsieh, Whole Earth Catalog, why are manhole covers round?, zero-sum game

Bezos himself bought into the Web’s new orthodoxy of openness, preaching inside Amazon over the next few months that they should make these new tools available to developers and “let them surprise us.” The company held its first developer conference that spring and invited all the outsiders who were trying to hack Amazon’s systems. Now developers became another constituency at Amazon, joining customers and third-party sellers. And the new group, run by Colin Bryar and Rob Frederick, was given a formal name: Amazon Web Services. It was the trailhead of an extremely serendipitous path. Amazon Web Services, or AWS, is today in the business of selling basic computer infrastructure like storage, databases, and raw computing power. The service is woven into the fabric of daily life in Silicon Valley and the broader technology community. Startups like Pinterest and Instagram rent space and cycles on Amazon’s computers and run their operations over the Internet as if the high-powered servers were sitting in the backs of their own offices.

Various divisions of the U.S. government, such as NASA and the Central Intelligence Agency, are high-profile AWS customers as well. Though Amazon keeps AWS’s financial performance and profitability a secret, analysts at Morgan Stanley estimate that in 2012, it brought in $2.2 billion in revenue. The rise of Amazon Web Services brings up a few obvious questions. How did an online retailer spawn such a completely unrelated business? How did the creature that was originally called Amazon Web Services—the group working on the commerce APIs—evolve into something so radically different, a seller of high-tech infrastructure? Early observers suggested that Amazon’s retail business was so seasonal—booming during the holiday months—that Bezos had decided to rent his spare computer capacity during the quieter periods.

Chapter 7: A Technology Company, Not a Retailer 1 Gary Rivlin, “A Retail Revolution Turns Ten,” New York Times, July 27, 2012. 2 Gary Wolf, “The Great Library of Amazonia,” Wired, October 23, 2003. 3 Ibid. 4 Luke Timmerman, “Amazon’s Top Techie, Werner Vogels, on How Web Services Follows the Retail Playbook,” Xconomy, September 29, 2010. 5 Shobha Warrier, “From Studying under the Streetlights to CEO of a U.S. Firm!,” Rediff, September 1, 2010. 6 Tim O’Reilly, “Amazon Web Services API,” July 18, 2002, http://www.oreillynet.com/pub/wlg/1707. 7 Damien Cave, “Losing the War on Patents,” Salon, February 15, 2002. 8 O’Reilly, “Amazon Web Services API.” 9 Steve Grand, Creation: Life and How to Make It (Darby, PA: Diane Publishing, 2000), 132. 10 Hybrid machine/human computing arrangement patent filed October 12, 2001; http://www.google.com/patents/US7197459. 11 “Artificial Artificial Intelligence,” Economist, June 10, 2006. 12 Katharine Mieszkowski, “I Make $1.45 a Week and I Love It,” Salon, July 24, 2006. 13 Jason Pontin, “Artificial Intelligence, with Help from the Humans,” New York Times, March 25, 2007. 14 Jeff Bezos, interview by Charlie Rose, Charlie Rose, PBS, February 26, 2009.


Terraform: Up and Running: Writing Infrastructure as Code by Yevgeniy Brikman

Amazon Web Services, cloud computing, DevOps, en.wikipedia.org, full stack developer, general-purpose programming language, microservices, Ruby on Rails

It was a dark and fearful age: fear of downtime, fear of accidental misconfiguration, fear of slow and fragile deployments, and fear of what would hap‐ pen if the sysadmins fell to the dark side (i.e. took a vacation). The good news is that thanks to the DevOps Rebel Alliance, there is now a better way to do things: Terra‐ form. Terraform is an open source tool that allows you to define your infrastructure as code using a simple, declarative programming language, and to deploy and manage that infrastructure across a variety of cloud providers (including Amazon Web Services, Azure, Google Cloud, DigitalOcean, and many others) using a few commands. For example, instead of manually clicking around a webpage or running dozens of com‐ mands, here is all the code it takes to configure a server: resource "aws_instance" "example" { ami = "ami-40d28157" instance_type = "t2.micro" } And to deploy it, you just run one command: > terraform apply Thanks to its simplicity and power, Terraform is rapidly emerging as a dominant player in the DevOps world.

My goal is to share what I’ve learned so you can avoid that lengthy process and become fluent in a matter of hours. Of course, you can’t become fluent just by reading. To become fluent in French, you’ll have to spend time talking with native French speakers, watching French TV shows, and listening to French music. To become fluent in Terraform, you’ll have to write real Terraform code, use it to manage real software, and deploy that software on real servers (don’t worry, all the examples use Amazon Web Service’s free tier, so it shouldn’t cost you anything). Therefore, be ready to read, write, and execute a lot of code. What you will find in this book Here’s an outline of what the book covers: Chapter 1, Why Terraform How DevOps is transforming the way we run software; an overview of infra‐ structure as code tools, including configuration management, orchestration, and server templating; the benefits of infrastructure as code; a comparison of Terra‐ form, Chef, Puppet, Ansible, Salt Stack, and CloudFormation.

Therefore, this book is focused on what the documentation does not cover: namely, how to go beyond introductory examples and use Terraform in a real-world setting. My goal is to get you up and running quickly by discussing why you may want to use Terraform in the first place, how to fit it into your workflow, and what practices and patterns tend to work best. x | Preface To demonstrate these patterns, I’ve included a number of code examples. Just about all of them are built on top of Amazon Web Services (AWS), although the basic pat‐ terns should be largely the same no matter what cloud provider you’re using. My goal was to make it as easy as possible for you to try the examples at home. That’s why the code examples all focus on one cloud provider (you only have to register with one service, AWS, which offers a free tier for the first year). That’s why I omitted just about all other third party dependencies in the book (including HashiCorp’s paid ser‐ vice, Atlas).


pages: 56 words: 16,788

The New Kingmakers by Stephen O'Grady

AltaVista, Amazon Web Services, barriers to entry, cloud computing, correlation does not imply causation, crowdsourcing, David Heinemeier Hansson, DevOps, Jeff Bezos, Khan Academy, Kickstarter, Marc Andreessen, Mark Zuckerberg, Netflix Prize, Paul Graham, Ruby on Rails, Silicon Valley, Skype, software as a service, software is eating the world, Steve Ballmer, Steve Jobs, Tim Cook: Apple, Y Combinator

Even with a growing portfolio of high-quality open source software available to them, developers remained limited by the availability of hardware. As creative as they could now be with their software infrastructure, to build anything of size, they would eventually have to procure hardware. This meant either purchasing it outright or renting it, typically for the minimum of a month, with the attendant set up, management, and maintenance fees on top. Enter Amazon Web Services (AWS). The idea was simple. Driven relentlessly by Moore’s Law, hardware doubled in speed every two years. Like Google and other Internet giants, Amazon discovered early that the most-economical model for scaling its technology was on cheap, commodity servers deployed by the hundreds or thousands. Having acquired the expertise to build, run, and manage these machines at scale, Amazon would leverage the same as a product.

Fielding, one the authors of HTTP, the protocol that still powers the Internet today—you’ve probably typed http:// many times yourself—advocated for a simple style that both reflected and leveraged the way the Internet itself had been constructed. Unsurprisingly, this simpler approach proved popular. At Amazon, for example, developers were able to choose between the two different mechanisms to access data: SOAP or REST. Even in 2004, 80% of those leveraging Amazon Web Services did so via REST. Two years after that, Google deprecated their SOAP API for search. And they were just the beginning. It’s hard to feel too sorry for SOAP’s creators, however—particularly if what one Microsoft developer told Tim O’Reilly is true: “It was actually a Microsoft objective to make [SOAP] sufficiently complex that only the tools would read and write this stuff,” he explained, “and not humans.”

Developers are attracted to its platform because of the size of the market…those developers create thousands of new applications…the new applications give consumers thousands of additional reasons to buy Apple devices rather than the competition…and those new Apple customers give even more developers reason to favor Apple. Apple not only profits from this virtuous cycle, it benefits from ever-increasing economies of scale, realizing lower component costs than competitors. None of which would be possible without the developers Apple has recruited and, generally, retained. Amazon Web Services The company that started the cloud computing craze was founded in 1994 as a bookstore. The quintessential Internet company, Amazon.com competed with the traditional brick-and-mortar model via an ever-expanding array of technical innovations: some brilliant, others mundane. The most-important of Amazon’s retail innovations co-opted its customers into contributors. From affiliate marketing programs to online reviews, Amazon used technology to enable its customers’ latent desire to more fully participate in the buying process.


pages: 472 words: 117,093

Machine, Platform, Crowd: Harnessing Our Digital Future by Andrew McAfee, Erik Brynjolfsson

"Robert Solow", 3D printing, additive manufacturing, AI winter, Airbnb, airline deregulation, airport security, Albert Einstein, Amazon Mechanical Turk, Amazon Web Services, artificial general intelligence, augmented reality, autonomous vehicles, backtesting, barriers to entry, bitcoin, blockchain, British Empire, business cycle, business process, carbon footprint, Cass Sunstein, centralized clearinghouse, Chris Urmson, cloud computing, cognitive bias, commoditize, complexity theory, computer age, creative destruction, crony capitalism, crowdsourcing, cryptocurrency, Daniel Kahneman / Amos Tversky, Dean Kamen, discovery of DNA, disintermediation, disruptive innovation, distributed ledger, double helix, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Ethereum, ethereum blockchain, everywhere but in the productivity statistics, family office, fiat currency, financial innovation, George Akerlof, global supply chain, Hernando de Soto, hive mind, information asymmetry, Internet of things, inventory management, iterative process, Jean Tirole, Jeff Bezos, jimmy wales, John Markoff, joint-stock company, Joseph Schumpeter, Kickstarter, law of one price, longitudinal study, Lyft, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, Marc Andreessen, Mark Zuckerberg, meta analysis, meta-analysis, Mitch Kapor, moral hazard, multi-sided market, Myron Scholes, natural language processing, Network effects, new economy, Norbert Wiener, Oculus Rift, PageRank, pattern recognition, peer-to-peer lending, performance metric, plutocrats, Plutocrats, precision agriculture, prediction markets, pre–internet, price stability, principal–agent problem, Ray Kurzweil, Renaissance Technologies, Richard Stallman, ride hailing / ride sharing, risk tolerance, Ronald Coase, Satoshi Nakamoto, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Skype, slashdot, smart contracts, Snapchat, speech recognition, statistical model, Steve Ballmer, Steve Jobs, Steven Pinker, supply-chain management, TaskRabbit, Ted Nelson, The Market for Lemons, The Nature of the Firm, Thomas Davenport, Thomas L Friedman, too big to fail, transaction costs, transportation-network company, traveling salesman, Travis Kalanick, two-sided market, Uber and Lyft, Uber for X, uber lyft, ubercab, Watson beat the top human players on Jeopardy!, winner-take-all economy, yield management, zero day

Kramer, “The Biggest Thing Amazon Got Right: The Platform,” Gigaom, October 12, 2011, https://gigaom.com/2011/10/12/419-the-biggest-thing-amazon-got-right-the-platform. 142 Dalzell was described as a bulldog: Matt Rosoff, “Jeff Bezos ‘Makes Ordinary Control Freaks Look like Stoned Hippies,’ Says Former Engineer,” Business Insider, October 12, 2011, http://www.businessinsider.com/jeff-bezos-makes-ordinary-control-freaks-look-like-stoned-hippies-says-former-engineer-2011-10. 143 launched Amazon Web Services in 2006: Amazon Web Services, “About AWS,” accessed February 4, 2017, https://aws.amazon.com/about-aws. 143 Amazon S3: Amazon Web Services, “Amazon Simple Storage Service (Amazon S3)—Continuing Successes,” July 11, 2006, https://aws.amazon.com/about-aws/whats-new/2006/07/11/amazon-simple-storage-service-amazon-s3---continuing-successes. 143 Amazon EC2: Amazon Web Services, “Announcing Amazon Elastic Compute Cloud (Amazon EC2)—Beta,” August 24, 2006, https://aws.amazon.com/about-aws/whats-new/2006/08/24/announcing-amazon-elastic-compute-cloud-amazon-ec2---beta. 143 more than 290,000 developers using the platform: Amazon, “Ooyala Wins Amazon Web Services Start-up Challenge, Receives $100,000 in Cash and Services Credits Plus Investment Offer from Amazon.com,” December 7, 2007, http://phx.corporate-ir.net/phoenix.zhtml?

The project was highly successful, and Amazon soon realized that it possessed a powerful new resource: a modular set of digital resources (like storage space, databases, and processing power) that could be combined and recombined almost at will—all accessible all over the world via the company’s existing high-speed Internet connections. Might these resources be valuable to people who wanted to build a database, application, website, or other digital resource but didn’t want to go through the trouble of maintaining all required hardware and software themselves? Amazon decided to find out and launched Amazon Web Services in 2006. It originally offered storage (Amazon S3) and computing (Amazon EC2) services on the platform. Within eighteen months, Amazon claimed to have more than 290,000 developers using the platform. Amazon Web Services added more tools and resources over time, maintained hardened interfaces, and kept growing dramatically. By April 2016, it was contributing 9% of Amazon’s total revenue and, remarkably, over half of the company’s total operating income. In early 2016, AWS was called the fastest-growing enterprise technology company in history by Deutsche Bank analyst Karl Keirstead.

Neural networks become much more powerful and capable as their size increases, and it’s only recently that sufficiently large ones have become cheap enough that they are available to many researchers. Cloud computing has helped open up AI research to these smaller budgets. Technology entrepreneur Elliot Turner estimates that the computing power required to execute a cutting-edge machine learning project could be rented from a cloud computing provider like Amazon Web Services for approximately $13,000 by the fall of 2016. Oddly enough, the popularity of modern video games has also been a great boost to machine learning. The specialized graphics processing units (GPUs) that drive popular gaming consoles turn out to be extremely well suited to the kinds of calculations required for neural networks, so they’ve been drafted in large numbers for this task. AI researcher Andrew Ng told us that “the teams at the leading edge do crazy complicated things in the GPUs that I could never imagine two or three years ago.”


pages: 116 words: 31,356

Platform Capitalism by Nick Srnicek

3D printing, additive manufacturing, Airbnb, Amazon Mechanical Turk, Amazon Web Services, Capital in the Twenty-First Century by Thomas Piketty, cloud computing, collaborative economy, collective bargaining, deindustrialization, deskilling, disintermediation, future of work, gig economy, Infrastructure as a Service, Internet of things, Jean Tirole, Jeff Bezos, knowledge economy, knowledge worker, liquidity trap, low skilled workers, Lyft, Mark Zuckerberg, means of production, mittelstand, multi-sided market, natural language processing, Network effects, new economy, Oculus Rift, offshore financial centre, pattern recognition, platform as a service, quantitative easing, RFID, ride hailing / ride sharing, Robert Gordon, self-driving car, sharing economy, Shoshana Zuboff, Silicon Valley, Silicon Valley startup, software as a service, TaskRabbit, the built environment, total factor productivity, two-sided market, Uber and Lyft, Uber for X, uber lyft, unconventional monetary instruments, unorthodox policies, Zipcar

These analytical divisions can, and often do, run together within any one firm. Amazon, for example, is often seen as an e-commerce company, yet it rapidly broadened out into a logistics company. Today it is spreading into the on-demand market with a Home Services program in partnership with TaskRabbit, while the infamous Mechanical Turk (AMT) was in many ways a pioneer for the gig economy and, perhaps most importantly, is developing Amazon Web Services as a cloud-based service. Amazon therefore spans nearly all of the above categories. Advertising Platforms The elders of this new enterprise form, advertising platforms are the initial attempts at building a model adequate to the digital age. As we will see, they have directly and indirectly fostered the emergence of the most recent technological trends – from the sharing economy to the industrial internet.

By all accounts, the Amazon Prime delivery service loses money on every order, and the Kindle e-book reader is sold at cost.34 On traditional metrics for lean businesses, this is unintelligible: unprofitable ventures should be cut off. Yet rapid and cheap delivery is one of the main ways in which Amazon entices users onto its platform in order to make revenues elsewhere. In the process of building a massive logistical network, Amazon Web Services (AWS) was developed as an internal platform, to handle the increasingly complex logistics of the company. Indeed, a common theme in the genesis of platforms is that they often emerge out of internal company needs. Amazon required ways to get new services up and running quickly, and the answer was to build up the basic infrastructure in a way that enabled new services to use it easily.35 It was quickly recognised that this could also be rented to other firms.

It also includes people who donate goods or purchase media online. An Intuit survey, on the other hand, reportedly found that 6% of the population in Britain is working in the sharing economy, but the actual data do not appear to be available. See Stokes, Clarence, Anderson, and Rinne, 2014: 25; Hesse, 2015. 76. Henwood, 2015. 77. Berg, 2016. 78. Knight, 2016. 79. See many more examples at Amazon Web Services, 2016. 80. Huet, 2016. 81. Ibid. 82. While government surveillance is often the focus of public attention today, corporate surveillance is just as pernicious a phenomenon. Pasquale, 2015. 83. ‘Reinventing the Deal’, 2015. 84. CB Insights, 2015. 85. Ibid. 86. CB Insights, 2016a. 87. National Venture Capital Association, 2016: 9; Crain, 2014: 374. 88. CB Insights, 2016d. 89. O’Keefe and Jones, 2015. 90.


pages: 313 words: 75,583

Ansible for DevOps: Server and Configuration Management for Humans by Jeff Geerling

AGPL, Amazon Web Services, cloud computing, continuous integration, database schema, Debian, defense in depth, DevOps, fault tolerance, Firefox, full text search, Google Chrome, inventory management, loose coupling, microservices, Minecraft, MITM: man-in-the-middle, Ruby on Rails, web application

If you’re interested in the current status of this effort, or would like to help in migrating to the v2 API, visit the v2 API support issue on GitHub. Dynamic inventory with AWS Many of this book’s readers are familiar with Amazon Web Services (especially EC2, S3, ElastiCache, and Route53), and likely have managed or currently manage an infrastructure within Amazon’s cloud. Ansible has very strong support for managing AWS-based infrastructure, and includes a dynamic inventory script to help you run playbooks on your hosts in a variety of ways. There are a few excellent guides to using Ansible with AWS, for example: Ansible - Amazon Web Services Guide Ansible for AWS I won’t be covering dynamic inventory in this chapter, but will mention that the ec2.py dynamic inventory script, along with Ansible’s extensive support for AWS infrastructure through ec2_* modules, makes Ansible the best and most simple tool for managing a broad AWS infrastructure.

If you’re daring, and want to test this feature, just log into your DigitalOcean account, delete one of the droplets just created by this playbook (maybe one of the two app servers), then run the playbook again. Now that we’ve tested our infrastructure on DigitalOcean, we can destroy the droplets just as easily (change the state parameter in provisioners/digitalocean.yml to default to 'absent' and run $ ansible-playbook provision.yml again). Next up, we’ll build the infrastructure a third time—on Amazon’s infrastructure. Provisioner Configuration: Amazon Web Services (EC2) For Amazon Web Services, provisioning works slightly different. Amazon has a broader ecosystem of services surrounding EC2 instances, and for our particular example, we will need to configure security groups prior to provisioning instances. To begin, create aws.yml inside the provisioners directory and begin the playbook the same ways as with DigitalOcean: 1 --- 2 - hosts: localhost 3 connection: local 4 gather_facts: false EC2 instances use security groups as an AWS-level firewall (which operates outside the individual instance’s OS).

Ansible Examples Other resources Chapter 1 - Getting Started with Ansible Ansible and Infrastructure Management On snowflakes and shell scripts Configuration management Installing Ansible Creating a basic inventory file Running your first Ad-Hoc Ansible command Summary Chapter 2 - Local Infrastructure Development: Ansible and Vagrant Prototyping and testing with local virtual machines Your first local server: Setting up Vagrant Using Ansible with Vagrant Your first Ansible playbook Summary Chapter 3 - Ad-Hoc Commands Conducting an orchestra Build infrastructure with Vagrant for testing Inventory file for multiple servers Your first ad-hoc commands Discover Ansible’s parallel nature Learning about your environment Make changes using Ansible modules Configure groups of servers, or individual servers Configure the Application servers Configure the Database servers Make changes to just one server Manage users and groups Manage files and directories Get information about a file Copy a file to the servers Retrieve a file from the servers Create directories and files Delete directories and files Run operations in the background Update servers asynchronously, monitoring progress Fire-and-forget tasks Check log files Manage cron jobs Deploy a version-controlled application Ansible’s SSH connection history Paramiko OpenSSH (default) Accelerated Mode Faster OpenSSH in Ansible 1.5+ Summary Chapter 4 - Ansible Playbooks Power plays Running Playbooks with ansible-playbook Limiting playbooks to particular hosts and groups Setting user and sudo options with ansible-playbook Other options for ansible-playbook Real-world playbook: CentOS Node.js app server Add extra repositories Deploy a Node.js app Launch a Node.js app Node.js app server summary Real-world playbook: Ubuntu LAMP server with Drupal Include a variables file, and discover pre_tasks and handlers Basic LAMP server setup Configure Apache Configure PHP with lineinfile Configure MySQL Install Composer and Drush Install Drupal with Git and Drush Drupal LAMP server summary Real-world playbook: Ubuntu Apache Tomcat server with Solr Include a variables file, and discover pre_tasks and handlers Install Apache Tomcat 7 Install Apache Solr Apache Solr server summary Summary Chapter 5 - Ansible Playbooks - Beyond the Basics Handlers Environment variables Per-play environment variables Variables Playbook Variables Inventory variables Registered Variables Accessing Variables Host and Group variables group_vars and host_vars Magic variables with host and group variables and information Facts (Variables derived from system information) Local Facts (Facts.d) Variable Precedence If/then/when - Conditionals Jinja2 Expressions, Python built-ins, and Logic register when changed_when and failed_when ignore_errors Delegation, Local Actions, and Pauses Pausing playbook execution with wait_for Running an entire playbook locally Prompts Tags Summary Chapter 6 - Playbook Organization - Roles and Includes Includes Handler includes Playbook includes Complete includes example Roles Role scaffolding Building your first role More flexibility with role vars and defaults Other role parts: handlers, files, and templates Handlers Files and Templates Organizing more complex and cross-platform roles Ansible Galaxy Getting roles from Galaxy Using role requirements files to manage dependencies A LAMP server in six lines of YAML A Solr server in six lines of YAML Helpful Galaxy commands Contributing to Ansible Galaxy Summary Chapter 7 - Inventories A real-world web application server inventory Non-prod environments, separate inventory files Inventory variables host_vars group_vars Ephemeral infrastructure: Dynamic inventory Dynamic inventory with DigitalOcean DigitalOcean account prerequisites Connecting to your DigitalOcean account Creating a droplet with Ansible DigitalOcean dynamic inventory with digital_ocean.py Dynamic inventory with AWS Inventory on-the-fly: add_host and group_by Multiple inventory sources - mixing static and dynamic inventories Creating custom dynamic inventories Summary Chapter 8 - Ansible Cookbooks Highly-Available Infrastructure with Ansible Directory Structure Individual Server Playbooks Main Playbook for Configuring All Servers Getting the required roles Vagrantfile for Local Infrastructure via VirtualBox Provisioner Configuration: DigitalOcean Provisioner Configuration: Amazon Web Services (EC2) Summary ELK Logging with Ansible ELK Playbook Forwarding Logs from Other Servers Summary GlusterFS Distributed File System Configuration with Ansible Configuring Gluster - Basic Overview Configuring Gluster with Ansible Summary Mac Provisioning with Ansible and Homebrew Running Ansible playbooks locally Automating Homebrew package and app management Configuring Mac OS X through dotfiles Summary Docker-based Infrastructure with Ansible A brief introduction to Docker containers Using Ansible to build and manage containers Building a Flask app with Ansible and Docker Data storage container Flask container MySQL container Ship it!


pages: 276 words: 64,903

Built for Growth: How Builder Personality Shapes Your Business, Your Team, and Your Ability to Win by Chris Kuenne, John Danner

Airbnb, Amazon Web Services, Berlin Wall, Bob Noyce, business climate, call centre, cloud computing, disruptive innovation, don't be evil, Fall of the Berlin Wall, Gordon Gekko, Jeff Bezos, Kickstarter, Lean Startup, Mark Zuckerberg, pattern recognition, risk tolerance, Sand Hill Road, self-driving car, Silicon Valley, Steve Jobs, Steve Wozniak, supply-chain management, zero-sum game

They didn’t care about our solution, if we weren’t sleeping, if our people were unhappy. All they cared about was maximizing their return.” Explorer in Action The Bezos commitment to innovation inspires a level of inventiveness that does not occur in other companies. —Chris Pinkham, Amazon Web Services Chris Pinkham had just convinced the higher-ups at Amazon to let him return to his native South Africa for the birth of his first child and to continue supporting his work on a project that would ultimately become AWS (Amazon Web Services). Before this move, when he was working for Amazon in Seattle in 2003, his curiosity led him to wonder whether there was a way to create “an infrastructure service for the world.” He thought about the underlying problem: “the cost of maintaining a reliable, scalable infrastructure in a traditional multi-datacenter model.”

We share the stories of well-known entrepreneurs like Ben Cohen and Jerry Greenfield (of ice cream fame) and Jack Dorsey of Twitter and Square, in addition to lesser-known founders like Nate Morris, Grace Choi, and Steve Breitman, who have built noteworthy value in their own endeavors—in sanitation, cosmetics, and apartment laundries, respectively. We include corporate entrepreneurs such as Chris Pinkham, the inventor of the underlying technology that enabled Amazon Web Services; the late Charlie Cawley, who built MBNA, the credit card behemoth, out of a small department within Maryland National Bank; and legends like Norbert Berta, who created the caplet dosage form that helped the Tylenol brand recovery from the notorious 1982 poisonings. Using many of these examples, we illustrate how each Builder Type approaches the dynamic challenges at play in building a highly successful business—where does each create value, and where does each risk destroying it?

Ability to attract talent: In many ways, an Explorer can become a bit of a folk hero among his or her community of people drawn to a specific kind of problem. Explorers use the question “Is there a better way?” as a magnet to attract enormously talented and like-minded systems thinkers. Earlier in this chapter, we saw Chris Pinkham and Mark Bonfigli each attract the talent who could build, respectively, the foundation for Amazon Web Services and one of the fastest-growing digital marketing firms for auto dealers. In some ways your articulation of the problem and perhaps the beginning of a vision on how to solve it create the same force of attraction that Crusaders enjoy (see chapter 4). This gift allows you to recruit the followers you need to take on successively larger and more complex problems. Autocratic predisposition: You probably already intimidate many of your employees with your demanding standard of understanding how everything works and the speed with which you move from thought to action.


pages: 260 words: 67,823

Always Day One: How the Tech Titans Plan to Stay on Top Forever by Alex Kantrowitz

accounting loophole / creative accounting, Albert Einstein, AltaVista, Amazon Web Services, augmented reality, Automated Insights, autonomous vehicles, Bernie Sanders, Clayton Christensen, cloud computing, collective bargaining, computer vision, Donald Trump, drone strike, Elon Musk, Firefox, Google Chrome, hive mind, income inequality, Infrastructure as a Service, inventory management, iterative process, Jeff Bezos, job automation, Jony Ive, knowledge economy, Lyft, Mark Zuckerberg, Menlo Park, new economy, Peter Thiel, QR code, ride hailing / ride sharing, self-driving car, Silicon Valley, Skype, Snapchat, Steve Ballmer, Steve Jobs, Steve Wozniak, Tim Cook: Apple, uber lyft, wealth creators, zero-sum game

Inside Amazon, Bezos has developed a culture that empowers employees to invent and lets them run the thing they’ve created (another leadership principle: Ownership). The deeper you dig in, the more apparent it becomes that this culture, bolstered by Wall Street investors who don’t demand Amazon turn a profit, is what’s behind the company’s array of beloved products and services: Echo, Kindle, Prime, Amazon Web Services, and Amazon.com. It is, in no uncertain terms, Amazon’s competitive advantage. Meet Amazon’s Science Fiction Writers On June 9, 2004, at 6:02 P.M., Jeff Bezos banned PowerPoint at Amazon. Not one for subtlety, he delivered the news right in an email subject line: “No powerpoint presentations from now on,” he wrote to his senior leadership team. PowerPoint, Bezos understood, is a terrific selling tool, making mediocre ideas look great by dressing them up in bullet points and fancy templates.

In 2011, Amazon’s VP of pricing and promotions, Dilip Kumar, left his position in the retail organization to shadow Bezos for two years as his “technical adviser.” Amazonians covet the technical adviser position. Those who inhabit it attend every meeting Bezos takes, getting a chance to look at Amazon through his eyes, and often gain license to take a major swing when they’re done. Andy Jassy, Bezos’s first technical adviser, went on to found Amazon Web Services (AWS), Amazon’s cloud services division, which now brings in some $9 billion per quarter with Jassy as its CEO. Kumar—whose LinkedIn profile describes his time as Bezos’s technical adviser as “Quite possibly the best job that I’ve ever had!”—also set out to do something big upon completing his tour of duty. But while he was away from Amazon’s retail division, Project Yoda began automating pricing and promotions, his domain of expertise, freeing him up (or forcing him) to try something new.

You had to be an expert in deployment in the cloud,” Athey said. “You also had to be an expert in A/B testing platforms, and continuous improvement, and machine learning. Satya was an expert in all of those things.” When Nadella took over Server & Tools in 2011, he understood that simply providing servers and tools to companies building software for desktop machines wasn’t going to be viable. Watching Amazon Web Services’ early success, and reading his economics team’s analysis, Nadella decided that acting any slower would set Microsoft back. While working on Bing, Nadella learned how difficult it was to be a distant number-two player in a market, and he wasn’t eager to repeat the scenario. Despite the risks to the core Windows asset—not to mention a thriving Server & Tools business—Nadella made clear that Server & Tools under him would focus on enabling cloud computing.


pages: 411 words: 98,128

Bezonomics: How Amazon Is Changing Our Lives and What the World's Best Companies Are Learning From It by Brian Dumaine

activist fund / activist shareholder / activist investor, AI winter, Airbnb, Amazon Web Services, Atul Gawande, autonomous vehicles, basic income, Bernie Sanders, Black Swan, call centre, Chris Urmson, cloud computing, corporate raider, creative destruction, Danny Hillis, Donald Trump, Elon Musk, Erik Brynjolfsson, future of work, gig economy, Google Glasses, Google X / Alphabet X, income inequality, industrial robot, Internet of things, Jeff Bezos, job automation, Joseph Schumpeter, Kevin Kelly, Lyft, Marc Andreessen, Mark Zuckerberg, money market fund, natural language processing, pets.com, plutocrats, Plutocrats, race to the bottom, ride hailing / ride sharing, Sand Hill Road, self-driving car, shareholder value, Silicon Valley, Silicon Valley startup, Snapchat, speech recognition, Steve Jobs, Stewart Brand, supply-chain management, Tim Cook: Apple, too big to fail, Travis Kalanick, Uber and Lyft, uber lyft, universal basic income, wealth creators, web application, Whole Earth Catalog

In early 2019, he was the richest man in the world with a net worth of $160 billion, and he remained in that top spot even after giving his ex-wife, MacKenzie, a quarter of their jointly owned Amazon stock (worth at the time $38 billion) in a divorce settlement. The company he founded controlled, as of 2019, nearly 40 percent of all online retailing in the U.S. and is one of the largest e-tailers in Europe. Amazon has expanded its Prime membership program to seventeen countries, and the number of people who have signed up for the service globally has hit more than 150 million. Bezos built Amazon Web Services (AWS) into the world’s largest cloud computing company, and Prime Video into a streaming media giant nipping at the heels of Netflix, and he’s the driving force behind the Echo, a smart speaker with Alexa inside that sold nearly 50 million units in its first few years of existence. Throughout the 2010s, this profitable company grew at an average rate of 25 percent a year—an astounding performance for such a large corporation (as of 2018, it had $233 billion in annual revenues).

Although she’s had an Amazon Prime subscription since she was eighteen years old, she still feels an endorphin surge when she comes home to find a package on her doorstep sealed with Amazon-branded packing tape. After breakfast, Ella takes the subway to her office. For her work, she searches for Bluetooth keyboards; no surprise, Amazon has the best selection. She clicks twice and knows they’ll be at her desk the next or maybe even the same day if she really needs them fast. She backs up important company files on the cloud built by Amazon Web Services, researches small-business loans offered by Amazon Lending, then gathers her team to discuss her start-up’s next major milestone: launching a new product on the Amazon site. That evening, on her way home, she stops at a cashier-less Amazon Go store to pick up a snack, and when she leaves, sensors and cameras automatically charge her Amazon account for what she carries out. She returns home, where she asks Alexa to read her a recipe for dinner.

Along those lines, Amazon already has a deal with Lennar, the nation’s largest homebuilder, to preinstall Alexa in all the company’s new homes. Another way that Amazon invades a new industry is to take something that it does well internally and then offer that service to others. The computer expertise that Amazon acquired while selling books online was impressive. Why not share that capability with other businesses? In 2006, Amazon Web Services was born. After a decade of building web applications, Amazon realized it had developed a core competency in operating computer infrastructure and data centers at a massive scale and that it could offer cloud services for customers at a great price. Today, AWS is the largest cloud computing company in the world by a wide margin. And since the folks at AWS were also really good at AI and machine learning, why not sell that knowledge to customers at an attractive price?


pages: 286 words: 87,401

Blitzscaling: The Lightning-Fast Path to Building Massively Valuable Companies by Reid Hoffman, Chris Yeh

activist fund / activist shareholder / activist investor, Airbnb, Amazon Web Services, autonomous vehicles, bitcoin, blockchain, Bob Noyce, business intelligence, Chuck Templeton: OpenTable:, cloud computing, crowdsourcing, cryptocurrency, Daniel Kahneman / Amos Tversky, database schema, discounted cash flows, Elon Musk, Firefox, forensic accounting, George Gilder, global pandemic, Google Hangouts, Google X / Alphabet X, hydraulic fracturing, Hyperloop, inventory management, Isaac Newton, Jeff Bezos, Joi Ito, Khan Academy, late fees, Lean Startup, Lyft, M-Pesa, Marc Andreessen, margin call, Mark Zuckerberg, minimum viable product, move fast and break things, move fast and break things, Network effects, Oculus Rift, oil shale / tar sands, Paul Buchheit, Paul Graham, Peter Thiel, pre–internet, recommendation engine, ride hailing / ride sharing, Sam Altman, Sand Hill Road, Saturday Night Live, self-driving car, shareholder value, sharing economy, Silicon Valley, Silicon Valley startup, Skype, smart grid, social graph, software as a service, software is eating the world, speech recognition, stem cell, Steve Jobs, subscription business, Tesla Model S, thinkpad, transaction costs, transport as a service, Travis Kalanick, Uber for X, uber lyft, web application, winner-take-all economy, Y Combinator, yellow journalism

In contrast, “old economy” businesses often have low gross margins. Growing wheat is a low-margin business, as is selling goods in a store or serving food in a restaurant. One of the most amazing things about Amazon’s success is that it has been able to build a massive business based on retailing, which is generally a low-margin industry. And even Amazon now relies heavily on its high-margin SaaS business, Amazon Web Services (AWS). In 2016, AWS accounted for 150 percent of Amazon’s operating income, which means that the retail business actually lost money. Most of the valuable companies we’re focusing on in this book have gross margins of over 60, 70, or even 80 percent. In 2016, Google had a gross income of $54.6 billion on sales of $89.7 billion, for a gross margin of 61 percent. Facebook’s gross income was $23.9 billion on sales of $27.6 billion, for a gross margin of 87 percent.

Even with this strenuous effort, it took several years to “tame” the Fail Whale; it wasn’t until after Twitter made it through the 2012 US presidential election night without melting down that the company’s then–creative director Doug Bowman announced that the Great Blue Whale had been put to death. One of the main reasons for the very large increase in the growth of valuable Web companies that we’ve seen in recent years is Amazon’s cloud offering, Amazon Web Services (AWS), which has helped many such businesses navigate around infrastructure limitations. Dropbox, for example, was able to scale up its storage infrastructure much more quickly and easily because it used AWS storage, eliminating the need to build and maintain its own arrays of hard disks. AWS reflects one of the ways that Amazon has made operational scalability a competitive advantage.

Amazon began with books because this represented a large enough market with a product amenable to e-commerce (durable, fairly standard sizes, readily available through wholesale distributors). Since then, Amazon has steadily expanded from books into many other verticals, and today very nearly lives up to Bezos’s vision of an “everything store” (though you still can’t buy automobiles on Amazon…yet). Retail is a truly gargantuan market and Amazon has captured an almost unthinkable portion of it and even made its market much bigger by launching Amazon Web Services. Now, in addition to being “the everything store,” Amazon also provides much of the Internet’s computing power, bandwidth, and storage (including for other dominant companies like Netflix). Distribution Amazon was one of the first companies to fully grasp the possibilities of the Internet as a distribution platform in creating the first successful affiliate program, Amazon Associates, which incentivizes individuals and owners of other websites to refer customers to Amazon in exchange for a share of the revenues generated.


Data Wrangling With Python: Tips and Tools to Make Your Life Easier by Jacqueline Kazil

Amazon Web Services, bash_history, cloud computing, correlation coefficient, crowdsourcing, data acquisition, database schema, Debian, en.wikipedia.org, Firefox, Google Chrome, job automation, Nate Silver, natural language processing, pull request, Ronald Reagan, Ruby on Rails, selection bias, social web, statistical model, web application, WikiLeaks

Run the follow‐ ing command, but replace my-key-pair.pem with the name of your key pair and pub lic_dns_name with your public web address: ssh -i ~/.ssh/my-key-pair.pem_ ubuntu@_public_dns_name For example: Using Amazon Web Services | 471 ssh -i data-wrangling-test.pem ubuntu@ec2-12-34-56-128.compute-1.amazonaws.com When prompted with Are you sure you want to continue connecting (yes/no)? type in yes. At this point, your prompt will change slightly, showing you are in the console of the server you set up. You can now continue getting your server set up by getting your code onto the server and setting up automation to run on your machine. You can read more about deploying code to your new server in Chapter 14. To exit your server, type Ctrl-C or Cmd-C. Summary Now you have your first AWS server up and running. Use the lessons learned in Chapter 14 to deploy code to your server and run your data wrangling in no time! 472 | Appendix G: Using Amazon Web Services Index Symbols $ (Mac/Linux prompt), 12 %logstart command, 150 %save, 150 .bashrc, 446 .gitignore files, 211 .pem file, 471 =, 454-457 ==, 67, 454-457 > (Windows prompt), 12 >>> (Python prompt), 12 \ (escape), 96 A ActionChains, 324 addition, 32 Africa, data sources from, 133 agate library, 216-240 aggregate method, 238 Airbrake, 410 Amazon Machine Image (AMI), 470 Ansible, 399 APIs (application programming interfaces), 357-371 advanced data collection from Twitter's REST API, 364-367 advanced data collection from Twitter's streaming API, 368-370 challenges/benefits of using, 136 features, 358-362 keys and tokens, 360-362 rate limits, 358 REST vs. streaming, 358 simple data pull from Twitter's REST API, 362-364 tiered data volumes, 359 arguments, 47, 102 Asia, data sources from, 134 Atom, 15 Atom Shell commands, 428 attrib method, 63 audience, identifying, 248 autocompletion, Tab key for, 97 automation, 373-413 basic steps for, 375-377 command-line arguments for, 384 config files for, 381-384 distributed processing for, 392 email, 403-406 errors and issues, 377-378 large-scale, 397-400 local files, 380-381 logging, 401-403 logging as a service, 409 messaging, 403-409 monitoring of, 400-411 of operations with Ansible, 399 parallel processing for, 389-391 Python logging, 401-403 questions to clarify process, 375 queue-based (Celery), 398-399 reasons for, 373-375 script location, 378 sharing code with Jupyter notebooks, 397 simple, 393-397 special tools for, 379-393 uploading, 409 473 using cloud for data processing, 386-389 when not to automate, 411 with cron, 393-396 with web interfaces, 396 AWS (Amazon Web Services), 386, 396, 469-472 Amazon Machine Image, 470 launching a server, 469 logging into a server, 470-472 B backup strategies, 141 bad data, 167-173 bar chart, 250 bash, 425-432 commands, 433-437 executing files, 429 modifying files, 427-429 navigation from command line, 426 online resources, 432 searching with command line, 431-432 Beautiful Soup, 296-300 beginners, Python resources for, xiii, 5, 423 best practices, 197 bias, 247 binary mode, 47 blocks, indented, 48 blogs, 266 Bokeh, 254-257 Booleans, 19 Boston Python, 424 Bottle, 396 browser-based parsing, 313-331 screen reading with Ghost.py, 325-331 screen reading with Selenium, 314-325 built-in functions/methods, 459 built-in tools, 34-38 C C++, Python vs., 419 C, Python vs., 419 calling variables, 24 Canada, data sources from, 134 capitalization, 50-52 case sensitivity, 50-52 cat command, 431 cd command, 14, 50, 97, 427 Celery, 398-399 Central Asia, data sources from, 134 474 | Index charts/charting, 250-257 with Bokeh, 254-257 with matplotlib, 251-254 chat, automated messaging with, 406 chdir command, 433 chmod command, 430, 471 chown command, 430 cloud data storage, 147 for data processing automation, 386-389 using Git to deploy Python, 387-389 cmd, 432-437 code length of well-formatted lines, 106 saving to a file, 49 sharing with Jupyter, 268-272 whitespace in, 453 code blocks, indented, 48 code editor, 15 coding best practices, 197 command line bash-based, 425-432 making a file executable via, 205 navigation via, 425-437 running CSV data files from command line, 50-52 Windows CMD/PowerShell, 432-437 command-line arguments, automation with, 384 command-line shortcuts, 442 commands, 425-437 cat, 431 cd, 14, 50, 97, 427 chdir, 433 chmod, 430, 471 chown, 430 cp, 428 del, 434 dir, 35-36, 433, 436 echo, 434-432 find, 61, 432 history, 428, 432 if and fi, 442 ls, 50, 426-429, 440 make and make install, 430 move, 434 pwd, 14, 50, 426, 429 rm, 429 sudo, 14, 430 touch, 427 unzip, 431, 436 wget, 430 comments, 88 communications officials, 131 comparison operators, 454-457 config files, 381-384 containers, 276 copy method, 456 copyrights, 276 correlations, 232 counters, 81 cp command, 428 Crawl Spider, 334 cron, 393-396 crowdsourced data, 136 CSS (Cascading Style Sheets), 289-291, 304-311 CSV data, 44-52 importing, 46 running files from command line, 50-52 saving code to a file, 49 csv library, 46 cursor (class name), 365 D data CSV, 44-52 Excel, 73-90 formatting, 162-167 importing, 216-222 JSON, 52-55 machine-readable, 43-71 manual cleanup exercise, 121 from PDFs, 91-126 publishing, 264-272 saving, 192-195 XML, 55-70 data acquisition, 127-140 and fact checking, 128 case studies, 137-140 checking for readability, cleanliness, and longevity, 129 determining quality of data, 128 from US government, 132 locating sources for, 130-137 locating via telephone, 130 smell test for new data, 128 data analysis, 241-244 documenting conclusions, 245 drawing conclusions, 244 improving your skills, 416 searching for trends/patterns, 244 separating/focusing data, 242-243 data checking manual cleanup exercise, 121 manual vs. automated, 109 data cleanup, 149-189 basics, 150-189 determining right type of, 195 finding duplicates, 173-187 finding outliers/bad data, 167-173 fuzzy matching, 177-181 identifying values for, 151-162 normalizing, 191-192 reasons for, 149-189 regex matching, 181-186 replacing headers, 152-155 saving cleaned data, 192-195 scripting, 196-212 standardizing, 191-192 testing with new data, 212 working with duplicate records, 186-187 zip method, 155-162 data containers, 23-28 dictionaries, 27 lists, 25-27 variables, 23-25 data exploration, 215-245 creating groupings, 235-240 identifying correlations, 232 identifying outliers, 233-235 importing data for, 216-222 joining datasets, 227-232 statistical libraries for, 240 data presentation, 247-273 avoiding storytelling pitfalls, 247-250 charts, 250-257 images, 263 interactives, 262 maps, 258-262 publishing your data, 264-272 time-related data, 257 tools for, 264 video, 263 visualization, 250-264 with illustrations, 263 with Jupyter, 268-272 with words, 263 Index | 475 data processing, cloud-based, 386-389 data storage, 140-148 alternative approaches, 147 cloud storage, 147 in databases, 141-146 in simple files, 146 local storage, 147 locations for, 140 data types, 18-22 and methods, 28-34 capabilities of, 28-34 decimals, 21 dictionary methods, 33 floats, 20 integers, 19 list methods, 32 non-whole number types, 20-22 numerical methods, 31 string methods, 30 strings, 18 data wrangling defined, xii duties of wranglers, 415 databases, 141-146 MongoDB, 145 MySQL, 141-143 nonrelational, 144-146 NoSQL, 144 PostgreSQL, 143 relational, 141-144 setting up local database with Python, 145-146 SQL, 142-143 SQLite, 145-146 Datadog, 410 Dataset (wrapper library), 145 datasets finding, 3 joining, 227-232 standardizing, 191-192 datetime module, 164 debugging, 13, 461 decimal module, 21 decimals, 21 default function arguments, 457 default values, arguments with, 102 del command, 434 delimiters, 38 deprecation, 60 476 | Index dictionaries, 27 dictionary methods, 33 dictionary values method, 154 DigitalOcean, 396 dir command, 35-36, 433, 436 directory, for project-related content, 445 distributed processing, 392 Django, 396 DNS name, public, 471 documentation for script, 198-209 of conclusions, 245 DOM (Document Object Model), 282 Dropbox, 147 duplicate records, 173-177 finding, 173-177 fuzzy matching, 177-181 regex matching, 181-186 working with, 186-187 E echo command, 434-432 Element objects, 61 ElementTree, 57 Emacs, 15 email, automation of, 403-406 emojis, 303 enumerate function, 158 errors, 228 escaping characters (\), 96 etree objects, 301 European Union, data sources from, 133 Excel installing Python packages for working with, 73 parsing files, 75-89 Python vs., 4 working with files, 73-90 except block, 228, 229 exception handling, 228 and logging, 410 catching multiple exceptions, 461 exception method, 402 extract method, 181 F Fabric, 397 Facebook chat, 408 fact checking, 128 files opening from different locations, 49 saving code to, 49 uncommon types, 124 find command, 61, 432 findall method, 61, 183 Flask, 396 floats, 20 FOIA (Freedom of Information Act) requests, 132 folders, 44 for loops, 47 and counters, 81 closing, 48 nested, 80 format method, 162 formatting data, 162-167 Freedom of Information Act (FOIA) requests, 132 functions, 47 built-in, 459 default arguments, 457 magic, 466-468 writing, 101 fuzzy matching, 177-181 G GCC (GNU Compiler Collection), 439 get_config function, 405 get_tables function, 117, 229 Ghost, 267 Ghost.py, 325-331 GhostDriver, 324 GIL (Global Interpreter Lock), 454 Git, 211, 387-389 GitHub Pages, 267 global private variables, 205 Google API, 357 Google Chat, 408 Google Drive, 147 Google Slides, 264 government data from foreign governments, 133 from US, 132 groupings, creating, 235-240 H Hadoop, 147 Haiku Deck, 264 hashable values, 174 HDF (Hierarchical Data Format), 147 headers replacing, 152-155 zip method for cleanup, 155-162 headless browsers and Ghost.py, 328 and Selenium, 324 help method, 37 Heroku, 268, 396 Hexo, 268 Hierarchical Data Format (HDF), 147 HipChat, 407 HipLogging, 408 history command, 428, 432 Homebrew finding Homebrew, 440-443 installation, 440 telling system where to find, 440-443 HTML, Python vs., 420 HypChat, 407 I if and fi commands, 442 if not statements, 169 if statements, 67 if-else statements, 67 illustrations (visual data presentation), 263 images, 263 immutable objects, 460 implicitly_wait method, 322 import errors, 14 import statements, 58 importing data, 216-222 in method, 154 indented code blocks, closing, 48 index method, 158 indexing defined, 83 for Excel files, 83 lists, 66 India, data sources from, 134 inheritance, 333-333 innerHTML attribute, 320 installation (see setup) instance type, AWS, 470 integers, 19 interactives, 262 internal methods, 35 Index | 477 Java, Python vs., 419 JavaScript console and web page analysis, 289-293 jQuery and, 291-293 style basics, 289-291 JavaScript, Python vs., 420 Jekyll, 267 join method, 230 jQuery, 291-293 JSON data, 52-55 Jupyter, 268-272 (see also IPython) shared notebooks, 271 sharing automation code with, 397 sharing data presentation code with, 268-272 learning about new environment, 448-451 virtual environment testing, 447 virtualenv installation, 444 virtualenvwrapper installation, 446 list generators, 152 list indexes, 66 list methods, 32 lists, 25-27 and addition, 32 indexing, 66, 83 local files, automation with, 380-381 logging and exceptions, 410 and monitoring, 410 as a service, 409 for automation monitoring, 401-403 logging module, 402 Loggly, 410 Logstash, 410 ls command, 50, 426-429, 440 Luigi, 397 LXML and XPath, 304-311 features, 311 installing, 301 reading web pages with, 300-311 K M IPython, 465-468 (see also Jupyter) installing, 16, 466 magic functions, 466-468 reasons for using, 465 is (comparison operator), 455 iterators, 217 itersiblings method, 304 J key pair, AWS, 470 keys API, 360-362 in Python dictionary, 27 L lambda function, 224 latency, 353 legal issues, 276 libraries (packages), 465 (see also specific libraries, e.g.: xlutils library) defined, 46 for working with Excel files, 73, 75 math, 22 statistical, 240 line chart, 250 LinkedIn API, 357 Linux installing Python on, 7 478 | Index Mac OS X Homebrew installation, 440 installing Python on, 8 learning about new environment, 448-451 Python 2.7 installation, 443 telling system where to find Homebrew, 440-443 virtual environment testing, 447 virtualenv installation, 444 virtualenvwrapper installation, 446 Mac prompt ($), 12 machine-readable data, 43-71 CSV data, 44-52 file formats for, 43 JSON data, 52-55 XML data, 55-70 magic commands, 150 magic functions, 466-468 main function, 204 make and make install commands, 430 markup patterns, 304-311 match method (regex library), 183 math libraries, 22 MATLAB, Python vs., 420 matplotlib, 251-254 medical datasets, 136 Medium.com, 265 Meetup (website), 424 messaging, automation of, 403-409 methods, 47 built-in, 459 dictionary, 33 list, 32 numerical, 31 string, 30 Middle East, data sources from, 134 modules (term), 21 MongoDB, 145 monitoring, logging and, 410 move command, 434 moving files, 434 MySQL, 141-143 N NA responses, 169 nested for loop, 80 Network tabs, 286-288 networks, Internet, 351-354 New Relic, 410 newline characters, 99 Node.js, Python vs., 420 non-governmental organizations (NGOs), datasets from, 135 nonrelational databases, 144-146 nose, 213 NoSQL, 144 numbers, 19, 22 numpy library, 175, 240 O object-oriented programming (OOP), 23 objects changing immutable, 460 defining vs. modifying, 459 Observation elements, 61 Octopress, 268 OOP (object-oriented programming), 23 open function, 47 operations automation, 399 organizations, data from, 135 outliers in data cleanup, 167-173 in data exploration, 233-235 P packages (see libraries) parallel processing, 389-391 pdfminer, 97-114 PDFs, 91-126 converting to text, 96 opening/reading with slate, 93-96 parsing tools, 92 parsing with pdfminer, 97-114 parsing with Tabula, 122-124 problem-solving exercises, 115-124 programmatic approaches to parsing, 92-97 table extraction exercise, 116-121 things to consider before using data from, 91 Pelican, 268 PhantomJS, 324 pip, 14, 74 PostgreSQL, 143 PowerShell, 435-437 online resources, 437 searching with, 435-437 Prezi, 264 private key, AWS, 471 private methods, 35 process module, 180 prompt, Python vs. system, 12 public DNS name, 471 publishing data, 264-272 creating a site for, 266 on Medium, 265 on pre-existing sites, 265-266 on Squarespace, 265 on WordPress, 265 on your own blog, 266 one-click deploys for, 268 open source platforms for, 266 with Ghost, 267 with GitHub Pages, 267 with Jekyll, 267 with Jupyter, 268-272 pwd command, 14, 50, 426, 429 PyData, 424 pygal, 260 pylab charts, 253 Index | 479 PyLadies, 423 PyPI, 74 pyplot, 253 pytest, 213 Python advanced setup, 439-451 basics, 17-41 beginner's resources, xiii, 5, 423 choosing version of, 6 getting started with, 5-16 idiosyncrasies, 453-463 installation, 443 launching, 18 reasons for using, xi, 4 setup, 7-11 test driving, 11-14 version 2.7 vs. 3.4, 6 Python prompt (>>>), system prompt vs., 12 Q queue-based automation, 398-399 quote_plus method, 295 R R, Python vs., 420 range() function, 78 rate limits, 358 ratio function, 178 Read the Docs (website), 423 read-only files, 47 reader function, 54 regular expressions (regex), 96, 181-186 relational databases, 141-144 remove method, 156 removing files, 435 renaming files, 434 reports, automated uploading of, 409 requests, web page, 294-296 REST APIs advanced data collection from Twitter's, 364-367 simple data pull from Twitter's, 362-364 streaming APIs vs., 358 return statement, 102 rm command, 429 robots.txt file, 293, 355 Rollbar, 410 round-trip latency, 353 Ruby/Ruby on Rails, Python vs., 421 480 | Index Russia, data sources from, 134 S SaltStack, 397 scatter charts, 254 scatter method, 254 scientific datasets, 136 scope, 458 Scrapely, 342 Scrapy, 332-351 building a spider with, 332-341 crawl rules, 348-350 crawling entire websites with, 341-351 retry middleware, 351 screen reading, 313 scripting and network problems, 351-354 data cleanup, 196-212 documentation for, 198-209 search method, 183 Selenium and headless browsers, 324 refreshing content with, 351 screen reading with, 314-325 Selenium ActionChains, 324 Sentry, 410 separators, 38 setup advanced, 439-451 code editor, 15 directory for project-related content, 445 GCC installation, 439 Homebrew, 440-443 IPython, 16, 466 learning about new environment, 448-451 libraries (packages), 443 Mac, 8 pip, 14 Python, 7-11, 443 Python 2.7 installation, 443 sudo, 14 virtual environment testing, 447 virtualenv installation, 444 virtualenvwrapper installation, 445 virtualenvwrapper-win installation, 447 Windows, 7, 9-11 set_field_value method, 327 shortcuts, command-line, 442 slate library, 93-96 SleekXMPP, 408 slicing, 84 smell test, 128 SMS automation, 406 South America, data sources from, 134 Spark, 393 Spider class, 334 spiders, 331-351 building with Scrapy, 332-341 crawling entire websites with Scrapy, 341-351 defined, 277 SQLite, 145-146 Squarespace, 265 Stack Overflow (website), 423 stacked chart, 250 startproject command, 335 statistical libraries, 240 storytelling audience considerations, 248 avoiding pitfalls, 247-250 data-wrangling as, 1-4 deciding what story to tell, 248 improving your skills, 417 streaming APIs advanced data collection from Twitter's, 368-370 REST APIs vs., 358 strftime method, 167 string methods, 30 strings and addition, 32 data types, 18 format method, 162 storing numbers as, 19 strip method, 29 strptime method, 164 Sublime Text, 15 subtraction, 32 sudo command, 14, 430 syntax errors, 14 sys module, 385 system prompt, Python prompt vs., 12 T Tab key, autocompletion with, 97 table extraction exercise, 116-121 table functions (agate), 223-226 table joins, 230 Tabula, 122-124 tag attributes, 56, 304 tags, 55 target audience, identifying, 248 telephone messages, automating, 406 telephone, locating data via, 130 terminal development closing indented code blocks, 48 IPython, 468 text messages, automation for, 406 text, converting PDFs to, 96 time series data, 258 time-related data, 257 timeline data, 258 Timeline tabs, 286-288 token, API, 360-362 tools built-in, 34-38 dir, 35-36 help, 37 type, 34 touch command, 427 trademarks, 276 try block, 228 TSV, 44 tuples, 112 Twillo, 406 Twitter, 1 advanced data collection from REST API, 364-367 advanced data collection from streaming API, 368-370 creating API key/access token for, 360-362 simple data pull from REST API, 362-364 type checking, 461 type method, 34 U United Kingdom, data sources from, 133 unittest, 213 universities, datasets from, 135 unsupported code, 121 unzip command, 431, 436 upper method, 30 V Vagrant, 397 values, Python dictionary, 27 variables, 23-25, 461 Index | 481 version (Python), choosing, 6 Vi, 15 video, 263 Vim, 15 virtual environment learning about, 448-451 testing, 447 virtualenv, 444 virtualenvwrapper installation, 445 updating .bashrc, 446 virtualenvwrapper-win, 447 visualization of data, 250-264 charts, 250-257 images, 263 interactives, 262 maps, 258-262 time-related data, 257 video, 263 with illustrations, 263 with words, 263 voice message automation, 406 W web interfaces, 396 web page analysis, 278-294 and JavaScript console, 289-293 in-depth, 293 inspection of markup structure, 278-286 Timeline/Network tab analysis, 286-288 web pages reading with Beautiful Soup, 296-300 reading with LXML, 300-311 requests, 294-296 web scraping advanced techniques, 313-354 and network problems, 351-354 basics, 275-312 browser-based parsing, 313-331 ethical issues, 354 legal issues, 276, 354 reading web pages with Beautiful Soup, 296-300 482 | Index reading web pages with LXML, 300-311 screen reading with Ghost.py, 325-331 screen reading with Selenium, 314-325 simple text scraping, 276-278 web page analysis, 278-294 web page requests, 294-296 with Scrapy, 332-351 with spiders, 331-351 with XPath, 304-311 wget command, 430 where function, 224 whitespace, 38, 50-52, 453 Windows installing Python on, 7, 9-11 learning about new environment, 448-451 virtual environment testing, 447 virtualenv installation, 444 virtualenvwrapper-win installation, 447 Windows 8, 9-11 Windows command line, 432-437 executing files from, 435 modifying files from, 434 navigation, 433 online resources, 437 searching with, 435-437 Windows PowerShell, 435-437 Windows prompt (>), 12 WordPress, 265 wrapper libraries, 145 X xlrd library, 75-79 xlutils library, 75 xlwt library, 75 XML data, 55-70 XPath, 304-311 Z Zen of Python, 196 zip function, 105 zip method, for data cleanup, 155-162 About the Authors Jacqueline Kazil is a data lover.

Advanced Python Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 E. Python Gotchas. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 F. IPython Hints. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 G. Using Amazon Web Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469 Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473 x | Table of Contents Preface Welcome to Data Wrangling with Python! In this book, we will help you take your data skills from a spreadsheet to the next level: leveraging the Python programming language to easily and quickly turn noisy data into usable reports.

Shared Jupyter notebooks Now that you are familiar with using Jupyter notebooks, you can set one up to share your code with others using a shared server. This will allow others to access your notebook on the normal Internet (not just localhost, like the notebook run from your terminal). There are some great tutorials available on how to set up a notebook server using DigitalOcean, Heroku, Amazon Web Services, Google DataLab, or whatever server you’d like. Remember to use secure passwords with your notebook server to ensure your notebooks are being used only by those with the pass‐ word. This will keep your server and data safe. We recommend setting up a version control system like Git (explored in more depth in Chapter 14) for your Jupyter notebooks as well, so you can have a history of your notebooks on a daily or weekly basis.


pages: 293 words: 78,439

Dual Transformation: How to Reposition Today's Business While Creating the Future by Scott D. Anthony, Mark W. Johnson

activist fund / activist shareholder / activist investor, additive manufacturing, Affordable Care Act / Obamacare, Airbnb, Amazon Web Services, autonomous vehicles, barriers to entry, Ben Horowitz, blockchain, business process, business process outsourcing, call centre, Clayton Christensen, cloud computing, commoditize, corporate governance, creative destruction, crowdsourcing, death of newspapers, disintermediation, disruptive innovation, distributed ledger, diversified portfolio, Internet of things, invention of hypertext, inventory management, Jeff Bezos, job automation, job satisfaction, Joseph Schumpeter, Kickstarter, late fees, Lean Startup, Lyft, M-Pesa, Marc Andreessen, Mark Zuckerberg, Minecraft, obamacare, Parag Khanna, Paul Graham, peer-to-peer lending, pez dispenser, recommendation engine, self-driving car, shareholder value, side project, Silicon Valley, Skype, software as a service, software is eating the world, Steve Jobs, the market place, the scientific method, Thomas Kuhn: the structure of scientific revolutions, transfer pricing, uber lyft, Watson beat the top human players on Jeopardy!, Y Combinator, Zipcar

Many of these, such as Amazon Prime (free shipping for members) and customer reviews, fit within the rubric of transformation A, part of Amazon’s continual transformation from a retailer of books to achieve its vision of the world’s most customer-centered retailer. Other innovations push Amazon in new, disruptive directions. Perhaps the company’s most notable transformation B effort is the creation of Amazon Web Services (AWS), which has become the leading provider of cloud computing services. Imagine going back to 2005, when Amazon commissioned the work that ultimately spawned AWS, and predicting that a decade later the company would be the dominant market leader. People would have laughed. AWS began as an effort led by programmer John Dalzell to accelerate internal IT projects. The idea involved breaking systems into bite-sized blocks and using open source software to build a service that made it easy for software developers to run any solution on the Amazon servers.

Accelerate Capability Development via Acquisitions and External Hires Generally, a company pursuing transformation B needs to assess the new competitors it will encounter. If your competitive set isn’t changing, after all, it suggests you aren’t stretching far enough. For example, in its core retailing business, Amazon competes against other retailers, such as Walmart and Barnes & Noble. Its Amazon Web Services business has a completely different set of competitors, such as Infosys, Accenture, Microsoft, and IBM. It’s easy to discount the challenges of moving into new markets with a different set of competitors, particularly in a disruptive circumstance, when many of the competitors will be fast-moving entrants. Triumphing over these new competitors requires humbly recognizing the limits of your current capabilities and aggressively closing critical capability gaps.

Singapore Postal Services Statistics: Infocomm Development Authority of Singapore, http://www.ida.gov.sg/Policies-and-Regulations/Industry-and-Licensees/Standards-and-Quality-of-Service/Quality-of-Service/Postal-Services, accessed July 2, 2016. Simon Israel’s commitment to transformation and transparency: Speech at the 24th SingPost Annual General meeting, July 14, 2016, http://infopub.sgx.com/FileOpen/Chairman%20Speech.ashx?App=Announcement&FileID=412793. Creation of Amazon Web Services: Charles O’Reilly and Michael Tushman, Lead and Disrupt: How to Solve the Innovator’s Dilemma (Stanford, CA: Stanford University Press, 2016); Brad Stone, The Everything Store: Jeff Bezos and the Age of Amazon (New York: Little, Brown and Company, 2013). AWS market share: “AWS, Google, Microsoft and IBM pull away from pack in race for cloud market share,” Business Cloud News, April, 29, 2016, www.businesscloudnews.com/2016/04/29/aws-google-microsoft-and-ibm-pull-away-from-pack-in-race-for-cloud-market-share/.


pages: 255 words: 55,018

Architecting For Scale by Lee Atchison

Amazon Web Services, business process, cloud computing, continuous integration, DevOps, Internet of things, microservices, platform as a service, risk tolerance, software as a service, web application

If your application started small and has seen incredible growth (and is now suffering from some of the growing pains associated with that growth), you might be suffering from reduced reliability and reduced availability. If you struggle with managing technical debt and associated application failures, this book will provide guidance in reducing that technical debt to make your application able to handle larger scale more easily. Why I Wrote This Book After spending many years at Amazon building highly scaled applications in both the retail and the Amazon Web Services (AWS) worlds, I moved to New Relic, which was in the midst of hyper growth. The company felt the pain of needing the systems and processes required to manage highly scaled applications, but hadn’t yet fully developed the processes and disciplines to scale its application. At New Relic, I saw firsthand the struggles of a company going through the process of trying to scale, and realized that there were many other companies experiencing the same struggles every day.

Most of the time, your in-stock percentage is above the SLA (we say we are meeting our SLA). However, one time in late summer it dropped below our 80% SLA for a short period of time (we say we failed our SLA). Note In certain industries, businesses have contractual agreements with customers that require them to meet established SLAs, perhaps with financial or other consequences for failing to meet them. Amazon Web Services, for example, has SLAs with its customers, and in some cases provides financial compensation if they fail to meet those SLAs. For example, with Amazon EC2 instances, if AWS’s monthly uptime percentage falls below 99.95%, it gives a service credit of 10% to affected customers. If it falls below 99.0%, AWS gives a service credit of 30%. You can find more details of how AWS calculates this SLA and the credit at https://aws.amazon.com/ec2/sla/.

., The Pros and Cons of Resource Allocation Techniques Amazonraw cloud resource management, Raw Resource-Raw Resource RDS, Managed Resource (Server-Based) Amazon API Gateway, Mobile Backend Amazon DynamoDB, Allocated-Capacity Resource Allocation Amazon EC2, Raw Resourceas allocated capacity resource, Allocated-Capacity Resource Allocation AZ vs. data centers with, Availability Zones Are Not Data Centers-Availability Zones Are Not Data Centers monitoring and CloudWatch, Monitoring and CloudWatch SLAs, What are Service-Level Agreements? Amazon Elastic Load Balancer (ELB), Changing Allocations Amazon Kinesis, Internet of Things Data Intake Amazon S3, Managed Resource (Non-server-based)Lambda and, Event Processing limits of, The “Magic” of Usage-Based Resource Allocation usage-based allocation, The “Magic” of Usage-Based Resource Allocation Amazon Web Services (see AWS) API contracts, The Ownership Benefit API Gateway, Mobile Backend application availability (see availability) applicationsbuilding with failure in mind, Focus #1: Build with Failure in Mind building with scaling in mind, Focus #2: Always Think About Scaling distributing across the cloud, Distributing the Cloud-Maintaining Location Diversity for Availability Reasons effects of growth, Preface guidelines for separating into services, What Should Be a Service?


pages: 603 words: 141,814

Python for Unix and Linux System Administration by Noah Gift, Jeremy M. Jones

Amazon Web Services, bash_history, Bram Moolenaar, cloud computing, create, read, update, delete, database schema, Debian, distributed revision control, Firefox, Guido van Rossum, industrial robot, inventory management, job automation, Mark Shuttleworth, MVC pattern, skunkworks, web application

Bayer, Michael, SQLAlchemy ORM Bicking, Ian, virtualenv blocks of code, editing, Magic Edit bookmark command, bookmark bookmarks, cd navigating to bookmarked directories, cd bootstrapped virtual environment, custom, Creating a Custom Bootstrapped Virtual Environment Boto (Amazon Web services), Amazon Web Services with Boto Buildout tool, Buildout, Developing with Buildout, Developing with Buildout developing with, Developing with Buildout bzip2 compression, Using tarfile Module to Create TAR Archives C callbacks, Callbacks, Callbacks capitalization, Built-in methods for str data extraction (see case) case (capitalization), Built-in methods for str data extraction converting for entire string, Built-in methods for str data extraction cd command, cd, cd, cd, dhist -<TAB> option, dhist -b option, cd charts, creating, Graphical Images checksum comparisons, MD5 Checksum Comparisons, MD5 Checksum Comparisons choices usage pattern (optparse), Choices Usage Pattern close( ) function (socket module), socket close() method, Creating files close() method (shelve), shelve cloud computing, Cloud Computing, Building a sample Google App Engine application, Amazon Web Services with Boto, Google App Engine, Building a sample Google App Engine application Amazon Web services with Boto, Amazon Web Services with Boto cmp() function (filecmp module), Using the filecmp Module combining strings, Built-in methods for str data extraction command history, History command line, Introduction, Summary, Basic Standard Input Usage, Basic Standard Input Usage, Introduction to Optparse, Option with Multiple Arguments Usage Pattern, Unix Mashups: Integrating Shell Commands into Python Command-Line Tools, Hybrid Kudzu Design Pattern: Wrapping a Unix Tool in Python to Spawn Processes, Integrating Configuration Files, Integrating Configuration Files basic standard input usage, Basic Standard Input Usage, Basic Standard Input Usage integrating configuration files, Integrating Configuration Files, Integrating Configuration Files integrating shell commands, Unix Mashups: Integrating Shell Commands into Python Command-Line Tools, Hybrid Kudzu Design Pattern: Wrapping a Unix Tool in Python to Spawn Processes optparse, Introduction to Optparse, Option with Multiple Arguments Usage Pattern community of Python users, Why Python?

to obtain object information, pinfo ' (quotation mark, single), Creating strings creating strings with, Creating strings " (quotation marks, double), Creating strings creating strings with, Creating strings _ (underscore), rehash, psearch, psearch, History results, History results for results history, History results, History results __ (in variable names), rehash __ object, psearch ___ object, psearch “magic” functions, alias (see also specific function) A Active Directory, using with Python, Using LDAP with OpenLDAP, Active Directory, and More with Python, Importing an LDIF File active version of package, changing, Change Active Version of Package alias function, alias, alias, alias alias table, rehash, rehashx Amazon Web services (Boto), Amazon Web Services with Boto Apache config file, hacking (example), Apache Config File Hacking, Apache Config File Hacking Apache log reporting, Apache Log Reporting, Apache Log Reporting Apache Log Viewer, building (example), Building an Apache Log Viewer Using PyGTK, Building an Apache Log Viewer Using PyGTK, Building an Apache Log Viewer Using Curses, Building an Apache Log Viewer Using Curses, Apache Log Viewer Application, Apache Log Viewer Application with curses library, Building an Apache Log Viewer Using Curses, Building an Apache Log Viewer Using Curses with Django, Apache Log Viewer Application, Apache Log Viewer Application with PyGTK, Building an Apache Log Viewer Using PyGTK, Building an Apache Log Viewer Using PyGTK Apache logfile, parsing (example), Log Parsing, Log Parsing appscript project, OS X Scripting APIs archiving data, Archiving, Compressing, Imaging, and Restoring, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files examining TAR file contents, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files ARP protocol, Creating Hybrid SNMP Tools asr utility, Automatically Re-Imaging Machines attachments (email), sending, Sending attachments with Python attrib attribute (Element object), ElementTree authentication, Authenticating to a Password Protected Site when installing eggs, Authenticating to a Password Protected Site authentication (SMTP), Using SMTP authentication automated information gathering, Automated Information Gathering, Receiving Email, Receiving Email, Receiving Email receiving email, Receiving Email, Receiving Email automatically re-imaging routines, Automatically Re-Imaging Machines automation, with IPython shell, Automation and Shortcuts, rep B background threading, IPython and Net-SNMP backslash (\), Creating strings escape sequences, list of, Creating strings backups, Introduction, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files examining TAR file contents, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files bar charts, creating, Graphical Images Bash, Python versus, Why Python?

to obtain object information, pinfo ' (quotation mark, single), Creating strings creating strings with, Creating strings " (quotation marks, double), Creating strings creating strings with, Creating strings _ (underscore), rehash, psearch, psearch, History results, History results for results history, History results, History results __ (in variable names), rehash __ object, psearch ___ object, psearch “magic” functions, alias (see also specific function) A Active Directory, using with Python, Using LDAP with OpenLDAP, Active Directory, and More with Python, Importing an LDIF File active version of package, changing, Change Active Version of Package alias function, alias, alias, alias alias table, rehash, rehashx Amazon Web services (Boto), Amazon Web Services with Boto Apache config file, hacking (example), Apache Config File Hacking, Apache Config File Hacking Apache log reporting, Apache Log Reporting, Apache Log Reporting Apache Log Viewer, building (example), Building an Apache Log Viewer Using PyGTK, Building an Apache Log Viewer Using PyGTK, Building an Apache Log Viewer Using Curses, Building an Apache Log Viewer Using Curses, Apache Log Viewer Application, Apache Log Viewer Application with curses library, Building an Apache Log Viewer Using Curses, Building an Apache Log Viewer Using Curses with Django, Apache Log Viewer Application, Apache Log Viewer Application with PyGTK, Building an Apache Log Viewer Using PyGTK, Building an Apache Log Viewer Using PyGTK Apache logfile, parsing (example), Log Parsing, Log Parsing appscript project, OS X Scripting APIs archiving data, Archiving, Compressing, Imaging, and Restoring, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files examining TAR file contents, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files ARP protocol, Creating Hybrid SNMP Tools asr utility, Automatically Re-Imaging Machines attachments (email), sending, Sending attachments with Python attrib attribute (Element object), ElementTree authentication, Authenticating to a Password Protected Site when installing eggs, Authenticating to a Password Protected Site authentication (SMTP), Using SMTP authentication automated information gathering, Automated Information Gathering, Receiving Email, Receiving Email, Receiving Email receiving email, Receiving Email, Receiving Email automatically re-imaging routines, Automatically Re-Imaging Machines automation, with IPython shell, Automation and Shortcuts, rep B background threading, IPython and Net-SNMP backslash (\), Creating strings escape sequences, list of, Creating strings backups, Introduction, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files examining TAR file contents, Using a tarfile Module to Examine the Contents of TAR Files, Using a tarfile Module to Examine the Contents of TAR Files bar charts, creating, Graphical Images Bash, Python versus, Why Python? Bayer, Michael, SQLAlchemy ORM Bicking, Ian, virtualenv blocks of code, editing, Magic Edit bookmark command, bookmark bookmarks, cd navigating to bookmarked directories, cd bootstrapped virtual environment, custom, Creating a Custom Bootstrapped Virtual Environment Boto (Amazon Web services), Amazon Web Services with Boto Buildout tool, Buildout, Developing with Buildout, Developing with Buildout developing with, Developing with Buildout bzip2 compression, Using tarfile Module to Create TAR Archives C callbacks, Callbacks, Callbacks capitalization, Built-in methods for str data extraction (see case) case (capitalization), Built-in methods for str data extraction converting for entire string, Built-in methods for str data extraction cd command, cd, cd, cd, dhist -<TAB> option, dhist -b option, cd charts, creating, Graphical Images checksum comparisons, MD5 Checksum Comparisons, MD5 Checksum Comparisons choices usage pattern (optparse), Choices Usage Pattern close( ) function (socket module), socket close() method, Creating files close() method (shelve), shelve cloud computing, Cloud Computing, Building a sample Google App Engine application, Amazon Web Services with Boto, Google App Engine, Building a sample Google App Engine application Amazon Web services with Boto, Amazon Web Services with Boto cmp() function (filecmp module), Using the filecmp Module combining strings, Built-in methods for str data extraction command history, History command line, Introduction, Summary, Basic Standard Input Usage, Basic Standard Input Usage, Introduction to Optparse, Option with Multiple Arguments Usage Pattern, Unix Mashups: Integrating Shell Commands into Python Command-Line Tools, Hybrid Kudzu Design Pattern: Wrapping a Unix Tool in Python to Spawn Processes, Integrating Configuration Files, Integrating Configuration Files basic standard input usage, Basic Standard Input Usage, Basic Standard Input Usage integrating configuration files, Integrating Configuration Files, Integrating Configuration Files integrating shell commands, Unix Mashups: Integrating Shell Commands into Python Command-Line Tools, Hybrid Kudzu Design Pattern: Wrapping a Unix Tool in Python to Spawn Processes optparse, Introduction to Optparse, Option with Multiple Arguments Usage Pattern community of Python users, Why Python?


Designing Web APIs: Building APIs That Developers Love by Brenda Jin, Saurabh Sahni, Amir Shevat

active measures, Amazon Web Services, augmented reality, blockchain, business process, continuous integration, create, read, update, delete, Google Hangouts, if you build it, they will come, Lyft, MITM: man-in-the-middle, premature optimization, pull request, Silicon Valley, Snapchat, software as a service, the market place, uber lyft, web application, WebSocket

Network I/O Network bottlenecks in modern applications are frequently caused by dependencies on external services requiring API calls across data centers. CPU Inefficient code performing expensive computations is one of the common causes of CPU bottlenecks. Memory Memory bottlenecks typically occur when systems do not have sufficient RAM. Most of the cloud hosting providers offer solutions to measure these bottlenecks. If you’re on Amazon Web Services (AWS), you can use Amazon CloudWatch. Heroku has New Relic. And on Google, you can use Stackdriver to access metrics and get insights into health, performance, and availability. To pinpoint specific bottlenecks, you can monitor the categories we just listed, homing in on your most frequently called API methods. One of the most obvious symptoms of a bottleneck is high latency for response times.

Enterprise developers are used to SDKs and frameworks rather than using the raw APIs. Building a Developer Strategy | 149 Attribute Platform of choice Preferred development language and frameworks Common use cases Preferred means of communication Market size and geographical distribution Description Windows and Linux scripting, web developers, SharePoint or Confluence. Java, .NET. Many work with Amazon Web Services (AWS; e.g., AWS Step Functions), some integrate with Slack for reporting. Some developers are exploring Node.js. Internal use cases—approval processes (time off, expenses, general), reports, and looking up clients in the customer relationship management (CRM) and ticketing systems seem like the most requested set of use cases. Key challenges are that enterprise developers are under a lot of pressure from the business to implement a lot of processes.

API Implementation Checklist: ❏ Define specific developer problem to solve ❏ Write internal API specification ❏ Get internal feedback on API specification ❏ Build API ❏ Authentication ❏ Authorization ❏ Error handling ❏ Rate-limiting ❏ Pagination ❏ Monitoring and logging ❏ Write documentation ❏ Run beta test with partners on new API ❏ Gather feedback from beta partners and make changes ❏ Create communication plan to notify developers of changes ❏ Release API changes API Design Worksheets | 205 Index A additions to APIs, 131 additive-change strategy, 133 Amazon Web Services (AWS), 83 ambassador programs, 195 analytics dashboards, 56 Apache Thrift, 14 API testers, 179 APIs about, 1 attributes and traits of good APIs, 199 business case for, 3-7 APIs as a product, 6 APIs for external developers first, internal developers second, 5 APIs for internal developers first, external developers second, 4 characteristics of great APIs, 7 description language, 122-126 design paradigms, 9 designing (see designing APIs, best practices; designing APIs, practical exercise in) event-driven, 19-25 comparison of different types, 24 HTTP Streaming, 23-24 WebHooks, 19-22 WebSockets, 22-23 request–response comparison of types, 18 GraphQL, 14-18 REST, 10-13 RPC, 13-14 security (see security) uses of, 2 application names, misleading, pro‐ hibiting in OAuth, 40 application-level caching for APIs, 87 archiving a GitHub repository, HTTP request for, 12 asynchronous operations, 89 authentication, 27 choosing mechanism for MyFiles API (example), 66 authorization, 28 OAuth 2.0 as standard for, 29 Authorization header (HTTP) for Basic Authentication, 28 with OAuth tokens and scope, 34 automated testing (see testing) awareness tactics examples, 156 AWS (see Amazon Web Services) B backward compatibility, maintaining, 58, 127-128 Basic Authentication, 28 beta programs, 188-190 beta testers, 79 Botkit framework, 177 bottlenecks, finding, 82 207 breadth and depth analysis for devel‐ oper programs, 185 breadth developer programs, 192-197 bulk operation endpoints, support‐ ing, 95 business case for APIs, 3-7 business objectives, defining, 62-64 business-focused tech savvy audience, 145 C cache invalidation, 87 caching in developer SDKs, 115 using to scale thoughput, 87 change, managing, 117-142 aiming for consistency, 117-126 using automated testing, 120-126 backward compatibility, 127-128 planning for and communicating change, 128-141 additions to the API, 131 communication plan, 129 removing fields or endpoints, 132 versioning, 133-141 changelogs, 170 CI pipeline, 121 clickjacking, 40 client secret, ability to reset in OAuth, 39 cloud hosting providers, solutions for measuring bottlenecks, 83 code samples, 173 code snippets, 174 communication means, preferred, of developers, 149 communication plan for changes, 129 community contributions, 182-183 community, building, 192 computing resources, adding to scale applications, 85 consistency in an API, 50, 117-126 hallmarks of consistency, 118 using automated testing, 120-126 continuous integration (CI), 121 Conversations APIs (Slack), 13, 93 208 | Index CPU bottlenecks, 83 Create, Read, Update, and Delete operations (see CRUD operations) credit programs, 197 cross-site request forgery (CSRF), 27, 38 cross-site scripting (XSS), 27 CRUD operations, 10 HTTP verbs and REST conven‐ tions, 11 in MyFiles API (example), 67 in request–response API para‐ digms, 18 pros and cons of API paradigms for MyFiles API (example), 65 current actual numbers for developer funnel indicators, 155 cursor-based pagination, 99-101 advantages and disadvantages, 100 ID as cursor, 101 opaque strings as cursor, 101 timestamp as cursor, 101 custom HTTP headers rate-limit response headers, 110 with OAuth token and scope, 34 D dark launching rate-limiting, 109 data access patterns (new), introduc‐ ing in your API, 90 database indexes, 86 database profiling, 84 database replication, 85 database sharding, 85 date filters, 96 debugging tools, 179 DELETE method (HTTP), 10 (see also CRUD operations) depth developer programs, 187-192 design sprints, 160, 191 designing APIs, best practices, 47-59 design for great developer experi‐ ence, 48-59 making it fast and easy to get started, 48 making troubleshooting easy, 52-56 making your API extensible, 56 working toward consistency, 50 design for real-life use cases, 47-48 focusing on users, 2 designing APIs, practical exercise in, 61-79 scenario 1, 62-72 defining business objectives, 62-64 outlining key user stories, 64 selecting technology architec‐ ture, 65 writing an API specification, 68-72 scenario 2, 72-79 defining problem and impact statement, 73 getting feedback on the API spec, 77-79 selecting technology architec‐ ture, 74 writing an API specification, 74-77 developer ecosystem strategy, build‐ ing, 143-161 building a developer strategy, 147 defining your developer funnel, 152-154 funnel indicators, 154 deriving measurements, 160 developer segmentation, 147-150 common use cases and tasks, 148 examples of segmentation analysis, 149 identity, 147 market size and geographical distribution, 149 platform of choice, 148 preferred development lan‐ guage, framework, and tools, 148 preferred means of communi‐ cation, 149 proficiency, 148 developers, 144-147 business-focused tech savvy audience, 145 hackers, 145 hobbyist, 144 professional developers, 146 variations on categories men‐ tioned, 146 distilling the value proposition, 151 mapping current and future state, 155-156 outlining your tactics, 156-160 awareness tactics examples, 156 developer relations high-level quarterly plan, 159 proficiency tactics examples, 157 success tactics examples, 158 usage tactics examples, 157 developer programs, 185-198 breadth and depth analysis for, 185 breadth programs, 192-197 credit programs, 197 hackathons, 194 meetups and community, 192 online videos and streaming, 196 speaking at events and event sponsorships, 194 support, forums, and StackO‐ verflow, 196 train-the-trainer and ambassa‐ dor programs, 195 depth programs, 187-192 design sprints, 191 early access/beta program, 188 top partner program, 187 measuring, 197 developer relations, 143 Developer Relations core activities, 185 developer resources, 163-184 API documentation, 163-172 code samples and snippets, 172-175 Index | 209 community contribution, 182-183 development tools, 179 frameworks, 177 office hours, 181 rich media, 180 videos, 180 software development kits (SDKs), 175-177 webinars and online training, 182 developer SDKs (see software devel‐ opment kits) developers APIs for external developers first, internal developers second, 5 APIs for internal developers first, external developers second, 4 communicating with about API changes, 129 removal of fields or endpoints, 132 rate limits and, 110-112 trying APIs without signing up, 49 disk I/O, 83 documentation for APIs, 49, 163-172 changelog, 170 frequently asked questions, 168 Getting Started guides, 163 landing page, 169 reference documentation, 165 terms of service (ToS), 171 tutorials, 167 E early access/beta program, 188 edge caching, 87 error handling and exponential back‐ off in SDKs, 115 errors HTTP status codes in MyFiles API specification (example), 71 meaningful, 52-55 actionable errors and recom‐ mended error codes, 52 grouping errors into high-level categories, 53 210 | Index organizing into status codes, headers, and machinereadable and humanreadable codes, 54 event objects for MyFiles API techni‐ cal spec (example), 75 event-driven APIs, 19-25 comparison of different types, 24 HTTP Streaming, 23-24 pros and cons for MyFiles API (example), 74 WebHooks, 19-22 WebSockets, 22-23 events (developer), 194 Events API (Slack), 91 evolving API design, 90-97 adding new API methods, 92 best practices, 97 introducing new data access pat‐ terns, 90 new options to filter results, 95 supporting bulk endpoints, 95 explicit-version strategy, 134-138 exponential back-off, 115 extensibility of APIs, 56 F Facebook, ToS violations, 41 failures and retries (WebHooks), 20 feedback, getting on API specifica‐ tion, 77-79 fields in API responses, filtering, 96 filtering results, providing options for, 95 firewalls, WebHooks and, 21 fixed-window counter (rate-limiting), 107 Flannel (Slack), 88 forums, 196 frameworks, 177 frequently asked questions (FAQ), 168 function names, versioned, 136 G geographical distribution (develop‐ ers), 149 GET method (HTTP), 10 (see also CRUD operations) Getting Started guides, 163 GitHub, 5 addressing scalability challenges, 91 archiving a repository, HTTP request for, 12 OAuth scope headers in API response, 35 rate-limit response header, 111 rate-limiting at, 112 Gmail phishing attack on, 40 thin WebHook message payload, 45 Google Cloud Platform (GCP), 83 Google Developer Groups (GDG), 192 Google Hangouts, versioning case study, 140 GraphQL, 6, 14-18, 91 advantages over REST and RPC, 16 comparison to REST and RPC APIs, 18 Object Field deprecation, 133 pros and cons for MyFiles API (example), 66 gRPC, 14 gzip compression, using in SDKs, 114 custom rate-limit response head‐ ers X-RateLimit-Limit, 111 X-RateLimit-Remaining, 111 X-RateLimit-Reset, 111 organizing errors into, 54 specifying API versions in, 135 X-Frame-Options, 40 HTTP methods CRUD operations and REST con‐ ventions, 11 in REST APIs, 10 in REST, RPC, and GraphQL APIs, 18 in RPC-style APIs, 13 HTTP status codes description for errors in MyFiles API specification (example), 71 in REST APIs, 10 indicating redirection for moved/ moving resources, 135 organizing errors into, 54 returning for rate limits, 110 HTTP Streaming, 23-24 comparison with WebHooks and WebSockets, 24 pros and cons for MyFiles eventdriven API (example), 74 HTTPs endpoints (OAuth), 39 human-readable errors, 52 H I hackathons, 194 hackers, 145 hash-based message authentication code (HMAC), 43 Hello World exercise, 163 hobbyist developers, 144 horizontal scaling, 85 HTTP, 9 (see also request–response APIs) in RPC-style APIs, 13 HTTP headers custom OAuth headers X-Accepted-OAuth-Scopes, 35 X-OAuth-Scopes, 35 ID as cursor, 101 identity (developers), 147 iframes, rendering of authorization screen, disallowing, 40 impact statement for MyFiles API (example), 63 scenario 2, 73 indexes (database), 86 integrated development environ‐ ments (IDEs), 148 interface definition language, 122-126 interface description language (IDL), 122 interfaces, 1 Index | 211 J JavaScript object notation (JSON), 122 JSON responses in REST APIs, 11 JSON web APIs, 122-126 describing and validating requests, 125 describing and validating respon‐ ses, 123 K key indicators status report (devel‐ oper funnel), 155 key performance indicators (KPIs), connecting to developer activities, 160 L landing page for API documentation, 169 load testing, 84 logging changelog, 170 use in troubleshooting developer issues, 55 M machine-readable error codes, 52 Macys.com responsive checkout, 78 MAJOR, MINOR, and PATCH ver‐ sions, 137 managing change (see change, man‐ aging) market potential (developer funnel indicators), 155 market size, 149 measurements of developer activities, 160 measuring developer programs, 197 meetups and community, 192 memory bottlenecks, 83 methods, adding to APIs, 92 MINOR versions, 137 mocking data for interactive user testing, 78 Mutual TLS (Transport Layer Secu‐ rity), 44 212 | Index N network I/O, 83 noise (in WebHooks), 21 non-CRUD operations in REST APIs, 12 O OAuth, 28-42, 50 benefits of, 29 best practices, 38 listing and revoking authoriza‐ tions, 37 scopes, 32 Slack's move to granular OAuth scopes, 34 selection for use in MyFiles API (example), 66 token and scope validation, 34 token expiry and refresh tokens, 35 token generation, 30 objective key results (OKRs), 159 office hours, 181 offset-based pagination, 97 advantages and disadvantages, 98 opaque strings as cursor, 101 OpenAPI, 125 order filters, 96 P paginating APIs, 97-102 best practices, 102 cursor-based pagination, 99-101 advantages and disadvantages, 100 choosing cursor contents, 101 offset-based pagination, 97 advantages and disadvantages, 98 pagination support in developer SDKs, 114 partner engineers, 187 PATCH method (HTTP), 10 (see also CRUD operations) PATCH versions, 137 personally identifiable information (PII), 55 phishing attacks using misleading application names, 40 platform of choice (developers), 148 polling, 19 solving as API scaling problem in REST APIs, 90 WebHooks vs., 19 POST method (HTTP), 10 (see also CRUD operations) problem and impact statement for MyFiles API (example), 63 scenario 2, 73 professional developers, 146 proficiency (developer), 148 proficiency tactics examples, 157 profiling code, 83 programming languages, 148 implementing code snippets in, 174 PUT method (HTTP), 10 (see also CRUD operations) Q quarterly plan for developer relations, 159 R rate-limit response headers, 110 rate-limiting APIs, 102-114 best practices, 112 implementation strategies, 105-110 fixed-window counter, 107 sliding-window counter, 108 token bucket algorithm, 105 rate limits and developers, 110-112 documenting rate limits, 111 rate-limit status API, 111 rate-limiting policy, 103 Slack's rate-limiting, lessons learned from, 113 Stripe's rate-limiting strategies, 104 Read method, 10 (see also CRUD operations) read/write scopes, 32 Real-Time Messaging API, 91 reference apps, 173 reference documentation, 165 refresh tokens, 36 one-time-use, 39 remote procedure calls (RPCs), 13 (see also RPC APIs) removing endpoints or fields from APIs, 132 replay attacks, 43 request logs, providing for develop‐ ers, 55 requests adding request parameters to con‐ trol output, 131 describing and validating, 125 request parameters in version schemes, 136 request–response APIs, 9-19 comparison of different types, 18 GraphQL, 14-17 REST, 10-13 RPC, 13-14 resources (in REST APIs), 10 showing relationships between, 11 responses adding response fields, 131 describing and validating, 123-125 REST APIs, 10-13 comparison to RPC and GraphQL, 18 CRUD operations, HTTP verbs, and REST conventions, 11 general rules for, 10 non-CRUD operations, 12 payload creep, 17 polling as scaling problem, solv‐ ing, 90 pros and cons for MyFiles API (example), 66 showing relationships among resources, 11 retries (WebHooks), 20 rich media, 180-181 rich site summary (RSS) feed, adding to changelog, 170 RPC APIs, 13-14 Index | 213 comparison to REST and GraphQL, 18 general rules for, 13 HTTP request to Slack API, 13 pros and cons for MyFiles API (example), 66 Slack, Conversations APIs, 13 using protocols other than HTTP, 14 RSpec test using JSON Schema speci‐ fication, 125 S sandboxes and API testers, 179 scaling APIs, 81-116 evolving your API design, 90-97 adding new API methods, 92 best practices, 97 introducing new data access patterns, 90 providing new options to filter results, 95 supporting bulk endpoints, 95 providing developer SDKs, 114-116 caching frequently used data, 115 error handling and exponen‐ tial back-off, 115 pagination support, 114 rate-limiting support, 114 SDK best practices, 115 using gzip compression in SDKs, 114 scaling throughput, 82-90 adding computing resources, 85 best practices, 89 caching, 87 database indexes, 86 doing expensive operations asynchronously, 89 finding bottlenecks, 82 using pagination, 97-102 best practices, 102 cursor-based pagination, 99-101 offset-based pagination, 97 214 | Index using rate-limiting, 102-114 best practices, 112 implementation strategies, 105-110 rate limits and developers, 110-112 scopes (OAuth), 32 for sensitive information, 39 for use in MyFiles API (example), 66 in MyFiles API (example) scopes, operations, and resour‐ ces, 67 Slack's move to granular OAuth scopes, lessons learned from, 34 SDKs (see software development kits) search filters, 96 search operations in REST APIs, 12 security, 27-46 authentication and authorization, 27 for WebHooks, 20 OAuth, 28-42 best practices, 38 listing and revoking authoriza‐ tions, 37 scopes, 32 token and scope validation, 34 token expiry and refresh tokens, 35 token generation, 30 WebHooks, 42-45 semantic versioning specification (SemVer), 137 server-sent events (SSE), streaming data via, 24 short-lived authorization codes (OAuth), 39 short-term targets and market poten‐ tial (developer funnel indicators), 155 signatures (WebHook), 43 Slack APIs, 4 adding new API methods, 92 addressing scalability challenges with Events API, 91 API Metadata, 126 app credentials of Slack app with verification token, 42 changelog, 170 Conversations API, 132 supporting bulk operations, 95 developer segmentation for Slack, 149 early access/beta program, 189 Flannel, application-level edge cache, 87 inconsistency in, 118 long-lived tokens, 37 missing field on message pay‐ loads, 127 move to granular OAuth scopes, lessons learned from, 34 rate-limiting, lessons learned from, 113 RPC-style web API, 13 translation layer to maintain backward compatibility, 57 WebSocket-based real-time mes‐ saging API, 22 sliding-window counter (ratelimiting), 108 snippets (code), 174 software as a service (SaaS) compa‐ nies, 4 software development kits (SDKs), 50, 148, 175-177 developer SDKs, 114-116 best practices, 115 caching frequently used data, 115 error handling and exponen‐ tial back-off, 115 pagination support, 114 rate-limiting support, 114 using gzip compression, 114 maintaining, 178 SoundCloud API, value proposition, 151 speaking at developer events, 194 specification (spec), writing for an API, 68-72 MyFiles API WebHooks (exam‐ ple), 74-77 SQL databases, queries based on cur‐ sor values, 99 Stack Overflow, 196 Stackdrivers, 83 stakeholders, reviewing API specifi‐ cation with, 77 state parameter support (OAuth), 38 streaming, 196 Stripe online testing of API by develop‐ ers without signing up, 49 rate-limiting, 104 value proposition, 151 versioning case study, 139 subresources in APIs, 11 success tactics examples, 158 support for developers, 196 T task queues, 89 TCP (Transport Control Protocol), 22 technology architecture, selecting scenario 1 for MyFiles API (exam‐ ple), 65 scenario 2 for MyFiles API (exam‐ ple), 74 terms of service (ToS) violations of Facebook ToS, 41 writing, 171 testing automated, 120-126 describing and validating requests, 125 describing and validating responses, 123-125 sandboxes and API testers, 179 Thrift, 14 throughput, scaling, 82-90 adding computing resources, 85 best practices, 89 caching, 87 database indexes, 86 doing expensive operations asyn‐ chronously, 89 finding bottlenecks, 82 the Time to Hello World (TTHW), 164 Index | 215 timestamps, using as cursors, 101 TLS (Transport Layer Security), 44 token bucket algorithm (ratelimiting), 105 tokens (OAuth) and scope, validation of, 34 expiry and refresh tokens, 35 generation of, 30 Slack's long-lived tokens, 37 top partner program, 187 train-the-trainer and ambassador programs, 195 transformations between versions, 137 Transport Control Protocol (see TCP) transport patterns for MyFiles API (example), 65 troubleshooting, making easy for developers, 52-56 building tooling, 55 meaningful errors, 52-55 providing troubleshooting tools, 179 tutorials for APIs, 49, 167 Twitch, deprecation of an API, 59 Twitter, 90 cursor-based pagination, 99 U Uber developers and rate-limiting, 109 Unix timestamp as cursor, 99 Update method, 10 (see also CRUD operations) URI components, specifying versions in, 135 URIs, specification for MyFiles API (example), 70 usage tactics examples, 157 user interfaces (UIs), 88 user stories (key), outlining scenario 1 for MyFiles API (exam‐ ple), 64 scenario 2 for MyFiles API (exam‐ ple), 73 users, focusing on in API design, 2 216 | Index users.conversations API method (Slack), 94 V value proposition, distilling for your API, 151 verification tokens, 42 versioning APIs, 57, 133-141 additive-change strategy, 133 case study, Google Hangouts, 140 case study, Stripe, 139 explicit-version strategy, 134-138 policies for MAJOR and MINOR changes, 138 process management, 141 vertical scaling, 85 videos creating, 180 online videos and streaming, 196 W WebHooks, 19-22, 90 comparison with WebSockets and HTTP Streaming, 24 considerations for use in MyFiles API (example), 66 MyFiles API Webhooks Spec (example), 74-77 polling vs., 19 pros and cons for MyFiles eventdriven API (example), 74 security, 42-45 best practices, 45 Mutual Transport Layer Secu‐ rity, 44 request signing and WebHook signatures, 43 thin payloads and API retrieval, 44 verification tokens, 42 supporting, additional complexi‐ ties added by, 20 webinars and online training, 182 WebSockets, 22-23, 90 comparison with WebHooks and HTTP Streaming, 24 pros and cons for MyFiles eventdriven API (example), 74 Y YouTube API, value proposition, 151 X XML responses, REST APIs, 11 Index | 217 About the Authors Brenda Jin is an entrepreneur and software engineer.


pages: 292 words: 85,151

Exponential Organizations: Why New Organizations Are Ten Times Better, Faster, and Cheaper Than Yours (And What to Do About It) by Salim Ismail, Yuri van Geest

23andMe, 3D printing, Airbnb, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, Baxter: Rethink Robotics, Ben Horowitz, bioinformatics, bitcoin, Black Swan, blockchain, Burning Man, business intelligence, business process, call centre, chief data officer, Chris Wanstrath, Clayton Christensen, clean water, cloud computing, cognitive bias, collaborative consumption, collaborative economy, commoditize, corporate social responsibility, cross-subsidies, crowdsourcing, cryptocurrency, dark matter, Dean Kamen, dematerialisation, discounted cash flows, disruptive innovation, distributed ledger, Edward Snowden, Elon Musk, en.wikipedia.org, Ethereum, ethereum blockchain, game design, Google Glasses, Google Hangouts, Google X / Alphabet X, gravity well, hiring and firing, Hyperloop, industrial robot, Innovator's Dilemma, intangible asset, Internet of things, Iridium satellite, Isaac Newton, Jeff Bezos, Joi Ito, Kevin Kelly, Kickstarter, knowledge worker, Kodak vs Instagram, Law of Accelerating Returns, Lean Startup, life extension, lifelogging, loose coupling, loss aversion, low earth orbit, Lyft, Marc Andreessen, Mark Zuckerberg, market design, means of production, minimum viable product, natural language processing, Netflix Prize, NetJets, Network effects, new economy, Oculus Rift, offshore financial centre, PageRank, pattern recognition, Paul Graham, paypal mafia, peer-to-peer, peer-to-peer model, Peter H. Diamandis: Planetary Resources, Peter Thiel, prediction markets, profit motive, publish or perish, Ray Kurzweil, recommendation engine, RFID, ride hailing / ride sharing, risk tolerance, Ronald Coase, Second Machine Age, self-driving car, sharing economy, Silicon Valley, skunkworks, Skype, smart contracts, Snapchat, social software, software is eating the world, speech recognition, stealth mode startup, Stephen Hawking, Steve Jobs, subscription business, supply-chain management, TaskRabbit, telepresence, telepresence robot, Tony Hsieh, transaction costs, Travis Kalanick, Tyler Cowen: Great Stagnation, uber lyft, urban planning, WikiLeaks, winner-take-all economy, X Prize, Y Combinator, zero-sum game

They have transformed the world of newspapers and publishing. And they have profoundly changed the way we communicate and interact with one another. One reason for that change is that the cost of distributing a product or service, particularly if can be converted almost entirely to information, has dropped almost to zero. It used to require millions of dollars in servers and software to launch a software company. Thanks to Amazon Web Services (AWS), it now costs just a tiny fraction of that amount. Similar stories can be found in every department in every industry of the modern economy. History and common sense make clear that you cannot radically transform every part of an organization—and accelerate the underlying clock of that enterprise to hyper-speed—without fundamentally changing the nature of that organization. Which is why, over the last few years, a new organizational scheme congruent with these changes has begun to emerge.

It is our belief that ExOs will overwhelm traditional linear organizations in most industries because they take better advantage of the information-based externalities inaccessible to older structures, a feat that will empower them to grow faster—shockingly faster—than their linear counterparts, and then accelerate from there. It’s hard to pin down exactly when this new organizational form emerged. Various aspects of ExOs have been around for decades, but it is only over the last few years that they have really started to matter. If we had to pick an official ExO origin date, it would be March 2006, when Amazon launched Amazon Web Services and created the low-cost “Cloud” for medium and small businesses. From that date on, the cost of running a data center moved from a fixed CAPEX (Capital Expenditure) cost to a variable cost. Today, it is almost impossible to find a single startup that doesn’t use AWS. We have even found a simple metric that helps to identify and distinguish emerging Exponential Organizations: a minimum 10x improvement in output over four to five years.

In the case of counterexamples—such as Tesla owning its own factories or Amazon owning its own warehouses and local delivery services—the underlying reason isn’t financial; instead, the driving force is the scarcity of mission-critical resources involved, or that it’s so new that it’s now fully fleshed out. The information age now enables Apple and other companies to access physical assets anytime and anywhere, rather than requiring that they actually possess them. Technology enables organizations to easily share and scale assets not only locally, but also globally, and without boundaries. As we noted earlier, the launch of Amazon Web Services in March 2006 was a key inflection point in the rise of ExOs. The ability to lease on-demand computing that would scale on a variable cost basis utterly changed the IT industry. A new Silicon Valley phenomenon called TechShop is another example of this trend. In the same way that gyms use a membership model to aggregate expensive exercise machinery that few could afford to have at home, TechShop collects expensive manufacturing machinery and offers subscribers a small monthly fee ($125 to $175, depending on the location) for unlimited access to its assets.


pages: 421 words: 110,406

Platform Revolution: How Networked Markets Are Transforming the Economy--And How to Make Them Work for You by Sangeet Paul Choudary, Marshall W. van Alstyne, Geoffrey G. Parker

3D printing, Affordable Care Act / Obamacare, Airbnb, Alvin Roth, Amazon Mechanical Turk, Amazon Web Services, Andrei Shleifer, Apple's 1984 Super Bowl advert, autonomous vehicles, barriers to entry, big data - Walmart - Pop Tarts, bitcoin, blockchain, business cycle, business process, buy low sell high, chief data officer, Chuck Templeton: OpenTable:, clean water, cloud computing, connected car, corporate governance, crowdsourcing, data acquisition, data is the new oil, digital map, discounted cash flows, disintermediation, Edward Glaeser, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, financial innovation, Haber-Bosch Process, High speed trading, information asymmetry, Internet of things, inventory management, invisible hand, Jean Tirole, Jeff Bezos, jimmy wales, John Markoff, Khan Academy, Kickstarter, Lean Startup, Lyft, Marc Andreessen, market design, Metcalfe’s law, multi-sided market, Network effects, new economy, payday loans, peer-to-peer lending, Peter Thiel, pets.com, pre–internet, price mechanism, recommendation engine, RFID, Richard Stallman, ride hailing / ride sharing, Robert Metcalfe, Ronald Coase, Satoshi Nakamoto, self-driving car, shareholder value, sharing economy, side project, Silicon Valley, Skype, smart contracts, smart grid, Snapchat, software is eating the world, Steve Jobs, TaskRabbit, The Chicago School, the payments system, Tim Cook: Apple, transaction costs, Travis Kalanick, two-sided market, Uber and Lyft, Uber for X, uber lyft, winner-take-all economy, zero-sum game, Zipcar

Airbnb, “Host Protection Insurance,” https://www.airbnb.com/host-protection-insurance, accessed June 15, 2015; A. Cecil, “Uber, Lyft, and Other Rideshare Drivers Now Have Insurance Options,” Policy Genius, https://www.policygenius.com/blog/uber-lyft-and-other-rideshare-drivers-now-have-insurance-options/, accessed June 14, 2015. 48. Huckman, Pisano, and Kind, “Amazon Web Services.” 49. Jillian D’Onfro, “Here’s a Reminder Just How Massive Amazon’s Web Services Business Is,” Business Insider, June 16, 2014, http://www .businessinsider.com/amazon-web-services-market-share-2014-6. 50. Annabelle Gawer and Michael A. Cusumano, Platform Leadership: How Intel, Microsoft, and Cisco Drive Industry Innovation (Boston: Harvard Business School Press, 2002). 51. Adapted from Gawer and Cusumano, Platform Leadership. 52. Clarkson and Van Alstyne, “The Social Efficiency of Fairness.” 53.

Second, a platform ecosystem can evolve faster when the core platform is a clean, simple system rather than a tangle of numerous features. For this reason, C. Y. Baldwin and K. B. Clark of Harvard Business School describe a well-designed platform as consisting of a stable core layer that restricts variety, sitting underneath an evolving layer that enables variety.12 Today’s best-designed platforms incorporate this structural principle. For example, Amazon Web Services (AWS), the most successful platform for providing cloud-based information storage and management, focuses on optimizing a handful of basic operations, including data storage, computation, and messaging.13 Other services, which are used by just a fraction of AWS customers, are restricted to the periphery of the platform and provided through purpose-built apps. THE POWER OF MODULARITY There are advantages to an integral approach where the system is developed as quickly as possible to serve a single purpose, especially in the early days of a platform.

Bezos doesn’t care. 5. All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions. 6. Anyone who doesn’t do this will be fired. 7. Thank you; have a nice day! Astute application of this principle of transparency underlies the success of Amazon Web Services (AWS), the platform’s giant cloud services company. Andrew Jassy, Amazon’s vice president of technology, had observed how different divisions of Amazon kept having to develop web service operations to store, search, and communicate data.48 Jassy urged that these varied projects should be combined into a single operation with one clear, flexible, and universally comprehensible set of protocols.


pages: 391 words: 71,600

Hit Refresh: The Quest to Rediscover Microsoft's Soul and Imagine a Better Future for Everyone by Satya Nadella, Greg Shaw, Jill Tracie Nichols

"Robert Solow", 3D printing, Amazon Web Services, anti-globalists, artificial general intelligence, augmented reality, autonomous vehicles, basic income, Bretton Woods, business process, cashless society, charter city, cloud computing, complexity theory, computer age, computer vision, corporate social responsibility, crowdsourcing, Deng Xiaoping, Donald Trump, Douglas Engelbart, Edward Snowden, Elon Musk, en.wikipedia.org, equal pay for equal work, everywhere but in the productivity statistics, fault tolerance, Gini coefficient, global supply chain, Google Glasses, Grace Hopper, industrial robot, Internet of things, Jeff Bezos, job automation, John Markoff, John von Neumann, knowledge worker, Mars Rover, Minecraft, Mother of all demos, NP-complete, Oculus Rift, pattern recognition, place-making, Richard Feynman, Robert Gordon, Ronald Reagan, Second Machine Age, self-driving car, side project, Silicon Valley, Skype, Snapchat, special economic zone, speech recognition, Stephen Hawking, Steve Ballmer, Steve Jobs, telepresence, telerobotics, The Rise and Fall of American Growth, Tim Cook: Apple, trade liberalization, two-sided market, universal basic income, Wall-E, Watson beat the top human players on Jeopardy!, young professional, zero-sum game

But just a few years ago, that outcome seemed very doubtful. By 2008, storm clouds were gathering over Microsoft. PC shipments, the financial lifeblood of Microsoft, had leveled off. Meanwhile sales of Apple and Google smartphones and tablets were on the rise, producing growing revenues from search and online advertising that Microsoft hadn’t matched. Meanwhile, Amazon had quietly launched Amazon Web Services (AWS), establishing itself for years to come as a leader in the lucrative, rapidly growing cloud services business. The logic behind the advent of the cloud was simple and compelling. The PC Revolution of the 1980s, led by Microsoft, Intel, Apple, and others, had made computing accessible to homes and offices around the world. The 1990s had ushered in the client/server era to meet the needs of millions of users who wanted to share data over networks rather than on floppy disks.

In his annual letter to shareholders in April 2011, just as I was beginning my new role, Amazon CEO Jeff Bezos gleefully offered a short course on the computer science and economics underlying their burgeoning cloud enterprise. He wrote about Bayesian estimators, machine learning, pattern recognition, and probabilistic decision making. “The advances in data management developed by Amazon engineers have been the starting point for the architectures underneath the cloud storage and data management services offered by Amazon Web Services (AWS),” he wrote. Amazon was leading a revolution and we had not even mustered our troops. Years earlier I had left Sun Microsystems to help Microsoft capture the lead in the enterprise market, and here we were once again far behind. As a company, we’d been very publicly missing the mobile revolution, but we were not about to miss the cloud. I would miss working with colleagues at Bing, but I was excited to lead what I sensed would be the biggest transformation of Microsoft in a generation—our journey to the cloud.

It was time to move Azure into the mainstream of STB rather than have it be a side project. People, the human element of any enterprise, are ultimately the greatest asset, and so I set about assembling the right team, starting with Scott Guthrie, a very accomplished Microsoft engineer. He had spearheaded a number of successful company technologies focused on developers. I tapped him to lead engineering for Azure on its way to becoming Microsoft’s cloud platform—our answer to Amazon Web Services. Over time, many others from both inside and outside the company joined our effort. Jason Zander, another key leader who built .Net and Visual Studio, joined to lead the core Azure infrastructure. We recruited the highly regarded Big Data researcher Raghu Ramakrishnan from Yahoo and James Phillips who had cofounded the database company Couchbase. We relied heavily on the expertise of Joy Chik and Brad Anderson to advance our device management solutions for the mobile world.


pages: 161 words: 44,488

The Business Blockchain: Promise, Practice, and Application of the Next Internet Technology by William Mougayar

Airbnb, airport security, Albert Einstein, altcoin, Amazon Web Services, bitcoin, Black Swan, blockchain, business process, centralized clearinghouse, Clayton Christensen, cloud computing, cryptocurrency, disintermediation, distributed ledger, Edward Snowden, en.wikipedia.org, Ethereum, ethereum blockchain, fault tolerance, fiat currency, fixed income, global value chain, Innovator's Dilemma, Internet of things, Kevin Kelly, Kickstarter, market clearing, Network effects, new economy, peer-to-peer, peer-to-peer lending, prediction markets, pull request, QR code, ride hailing / ride sharing, Satoshi Nakamoto, sharing economy, smart contracts, social web, software as a service, too big to fail, Turing complete, web application

More likely, the blockchain infrastructure resembles a layer of cloud computing infrastructure. Blockchain virtual machines may be too expensive if we are to literally compare their functionality to a typical cloud service such as Amazon Web Services or DigitalOcean, but they will be be certainly useful for smart contracts that execute their logic on the blockchain’s virtual machinery, or decentralized applications, also called Dapps. As a sidenote, we could also see a future where client nodes can talk to each other directly in scenarios where blockchains are too expensive or slow. When you run an application in the cloud (for example, on Amazon Web Services or Microsoft Azure), you are billed according to a combination of time, storage, data transfer, and computing speed requirements. The novelty with virtual machine costing is that you are paying to run the business logic on the blockchain, which is otherwise running on physical servers (on existing cloud infrastructure), but you do not have to worry about setting up these servers because they are managed by other users who are getting paid anyways for running that infrastructure via mining.

There is magic when you figure out the blockchain’s touch points to your business and you start offering new user experiences that didn’t exist before. These new areas will include banking without banks, gambling without the house’s edge, title transfers without central authorities stamping them, e-commerce without eBay, registrations without government officials overseeing them, computer storage without Dropbox, transportation services without Uber, computing without Amazon Web Services, online identities without Google, and that list will continue to grow. Take any services and add “without previous center-based authority,” and replace with “peer-to-peer, trust-based network,” and you will start to imagine the possibilities. The general characteristics of decentralization-based services include: Speed in settlements No intermediary delays Upfront identification and reputation Flat structure with no overhead Permission-less user access Trust built inside the network Resiliency against attacks No censorship No central point of failure Governance decisions by consensus Peer-to-peer communications THE CRYPTO ECONOMY What started as Bitcoin, the poster child cryptocurrency that captured our imagination, is leading to a multiplicity of blockchain-enabled businesses and implementations.


pages: 90 words: 17,297

Deploying OpenStack by Ken Pepple

Amazon Web Services, cloud computing, database schema, Infrastructure as a Service, Kickstarter, Ruby on Rails, web application, x509 certificate

It is intended to provide the reader with a solid understanding of the OpenStack project goals, details of specific OpenStack software components, general design decisions, and detailed steps to deploy OpenStack in a few controlled scenarios. Along the way, readers would also learn common pitfalls in architecting, deploying, and implementing their cloud. Intended Audience This book assumes that the reader is familiar with public Infrastructure as a Service (IaaS) cloud offerings such as Rackspace Cloud or Amazon Web Services. In addition, it demands an understanding of Linux systems administration, such as installing servers, networking with iptables, and basic virtualization technologies. Conventions Used in This Book The following typographical conventions are used in this book: Italic Indicates new terms, URLs, email addresses, filenames, and file extensions. Constant width Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.

# nova-manage db sync # To view the database scheme version, use the db version arguments: # nova-manage db version 14 Note The database version for Cactus is 14 Instance Types and Flavors Instance types (or “flavors,” as the OpenStack API calls them) are resources granted to instances in Nova. In more specific terms, this is the size of the instance (vCPUs, RAM, Storage, etc.) that will be launched. You may recognize these by the names “m1.large” or “m1.tiny” in Amazon Web Services EC2 parlance. The OpenStack API calls these “flavors” and they tend to have names like “256 MB Server.” Instance types or flavors are managed through nova-manage with the instance_types command and an appropriate subcommand. At the current time, instance type manipulation isn’t exposed through the APIs nor the adminclient. Note You can use the flavor command as a synonym for instance_types in any of these examples.


pages: 309 words: 54,839

Attack of the 50 Foot Blockchain: Bitcoin, Blockchain, Ethereum & Smart Contracts by David Gerard

altcoin, Amazon Web Services, augmented reality, Bernie Madoff, bitcoin, blockchain, Blythe Masters, Bretton Woods, clean water, cloud computing, collateralized debt obligation, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, cryptocurrency, distributed ledger, Ethereum, ethereum blockchain, Extropian, fiat currency, financial innovation, Firefox, Flash crash, Fractional reserve banking, index fund, Internet Archive, Internet of things, Kickstarter, litecoin, M-Pesa, margin call, Network effects, peer-to-peer, Peter Thiel, pets.com, Ponzi scheme, Potemkin village, prediction markets, quantitative easing, RAND corporation, ransomware, Ray Kurzweil, Ross Ulbricht, Ruby on Rails, Satoshi Nakamoto, short selling, Silicon Valley, Silicon Valley ideology, Singularitarianism, slashdot, smart contracts, South Sea Bubble, tulip mania, Turing complete, Turing machine, WikiLeaks

As of March 2015, a full third of all Bitcoin exchanges up to then had been hacked, and nearly half had closed.80 Since the exchanges are largely uninsured, unregulated and not required to keep reserves, depositors’ money goes up in smoke. It’s not just scamminess on the part of the proprietors, but sheer jawdropping incompetence: Bitomat, then the third-largest exchange, were keeping the whole site’s wallet file on an Amazon Web Services EC2 server in the cloud that didn’t have separate backups and was set to “ephemeral,” i.e., it would disappear if you restarted it. Guess what happened in July 2011? Whoops.81 Bitcoinica was its sixteen-year-old creator’s first serious PHP project. He read up on PHP, Ruby on Rails, personal finance and startups, and wrote an exchange.82 It collapsed in May 2012: “No database backups … Everyone had root.”83 The exchange’s remaining funds were lost in further hacks, after the administrators turned out to be using their (leaked) Mt.

This is clearly superior, for a certain type of seller, to the IPO bubble of the dot-com era, in that these aren’t actually shares, and the purchasers have no influence over the funded enterprise even in theory. The ideas themselves are as bad as the worst dot-com IPOs. Digix, the first token crowdsale on the Ethereum blockchain itself, is a cryptocurrency backed by gold;309 Golem offers a “decentralized” (buzzword alert!) market in computing, like Amazon Web Services except you can only pay using their token;310 Gnosis offers semiautomatic prediction markets using their token;311 SingularDTV is a bizarre plan to fund a TV show about the Singularity in which a Caribbean island adopts Ethereum as its currency and Austrian economics works (this one gets its own section later in the book); Iconomi is an index fund of other ICOs.312 The token smart contracts are often incompetent in both intended functionality and programming ability.313 This turns out not to matter as long as they do the basic job: attract buyers and sell tokens.

He lives in east London with his spouse Arkady and their daughter. Until he reinstalled the laptop they were on, he was the proud owner of six Dogecoins. Index 8chan 53 A company for carrying on an undertaking of great advantage, but nobody to know what it is 100 Accenture 122 address 12 AIDS Trojan 72 AllCrypt 43 Alloway, Tracy 111 AlphaBay 72 AlphaPoint 84 altcoins 91 Altoid 51 Amazon Web Services 43, 98 anarcho-capitalism 18, 49 Andresen, Gavin 65 anonymity 26, 52 AntMiner 57 Apple 127, 137 arbitrage 83, 88 artificial intelligence 103, 117, 137 ASCAP 126 Ash, Jordan 96 ASIC 56 Astrobotic 94 Atlantis 54 Augur 104 Australian Securities Exchange 121 Australian Tax Office 63 Austrian economics 23, 49, 98, 137 Automattic 75 Ayre, Calvin 63 Azure Blockchain 122 b-money 19 Back, Adam 19 Bancor 95, 97 Bandcamp 130 banking the unbanked 29 baphomet (8chan) 53 BBC 32, 66 Beanie Baby 35 Berklee Rethink 128 BFX 85 Bitcoin creation of new bitcoins 14 economic aims 22 economic equality 30 exchanges 42 invention 20 irreversibility 26 limited supply 31 Satoshi Nakamoto (creator) 59 security 25 transactions per second 28 Bitcoin Bowl 77 Bitcoin Jesus 37 Bitcoin Mining Accidents 55 Bitcoin Relay Network 58 Bitcoin Savings & Trust 40 Bitcoinica 43, 83 bitcoinmarket.com 35 Bitcointalk 40, 41 Bitfinex 40, 83 BitGo 84 Bitgold 19 Bitmain 57 Bitomat 43 BitPay 43, 74 BitShares 99 BitTorrent 130, 134 Blem Information Management 116 blockchain 13 blockchain (business) 111 Blockchain for Creative Industries 132 Blockchain or the Chaingang?


pages: 340 words: 100,151

Secrets of Sand Hill Road: Venture Capital and How to Get It by Scott Kupor

activist fund / activist shareholder / activist investor, Airbnb, Amazon Web Services, asset allocation, barriers to entry, Ben Horowitz, carried interest, cloud computing, corporate governance, cryptocurrency, discounted cash flows, diversification, diversified portfolio, estate planning, family office, fixed income, high net worth, index fund, information asymmetry, Lean Startup, low cost airline, Lyft, Marc Andreessen, Myron Scholes, Network effects, Paul Graham, pets.com, price stability, ride hailing / ride sharing, rolodex, Sand Hill Road, shareholder value, Silicon Valley, software as a service, sovereign wealth fund, Startup school, Travis Kalanick, uber lyft, VA Linux, Y Combinator, zero-sum game

So I joined Credit Suisse First Boston and drank from the fire hose of the developing tech bubble. A few years into my job, on the eve of finishing an IPO for E.piphany, one of the marketing executives I had worked with to help them prepare for the IPO told me he was leaving to join a new startup called LoudCloud. Cofounded by Marc Andreessen, the already revered cofounder of Netscape, LoudCloud was trying to create a compute utility (much like Amazon Web Services has now created). Among the other cofounders was Ben Horowitz. This was the fall of 1999, and the dot-com excitement was in full swing. I had finally opened my eyes to what was happening around me, and I wanted to be a part of it. When my friend at E.piphany offered me the chance to meet Marc Andreessen and Ben Horowitz and see what they were doing, it was too much to pass up. My wife, who was about five months pregnant at the time with our first child and who was busy closing on the first house we were buying together, didn’t see it quite the same way.

Just as when you plug your phone charger into the wall socket you don’t need to know (or care) about how the electricity got there, you just use it, LoudCloud’s mission was to do the same for computing capacity. As an engineer, you should be able to develop your custom application and then just “plug it in” to the compute utility that could run the application seamlessly for you. You shouldn’t have to worry about what kind of database, networking equipment, application servers, etc., underlie the utility; it should simply work. It was a great idea—one that Amazon Web Services has built into a multibillion-dollar business today. LoudCloud was probably about ten years ahead of its time, an oft-repeated lesson, by the way, in the startup world. Though timing isn’t everything, timing is definitely something—it’s a big reason why we now see many ideas that failed in the dot-com bubble being reincarnated as successful businesses two decades later. As market conditions change—in the case of the dot-com businesses, the market size of available customers was simply too small relative to the cost of acquiring those customers—business models that previously failed can become viable.

First, starting in the early 2000s, the costs required to start a new company began to fall precipitously. As cloud computing began to take off, the unit costs of all these hardware and software products began to fall. A variant of Moore’s law was sweeping through every segment of the technology stack. At the same time, software development systems also progressed, and engineering efficiency increased correspondingly. Today, developers can go to Amazon Web Services or competing providers and rent compute utility on demand, providing incremental pricing coupled with dramatically lowered input costs. Thus the costs required to start a company have fallen significantly, and therefore the amount of money that startups need to raise at an early stage has declined accordingly. This is a good thing overall in that it means there is a lot of new company experimentation that can be had with only small amounts of actual capital put at risk.


pages: 404 words: 95,163

Amazon: How the World’s Most Relentless Retailer Will Continue to Revolutionize Commerce by Natalie Berg, Miya Knights

3D printing, Airbnb, Amazon Web Services, augmented reality, Bernie Sanders, big-box store, business intelligence, cloud computing, Colonization of Mars, commoditize, computer vision, connected car, Donald Trump, Doomsday Clock, Elon Musk, gig economy, Internet of things, inventory management, invisible hand, Jeff Bezos, market fragmentation, new economy, pattern recognition, Ponzi scheme, pre–internet, QR code, race to the bottom, recommendation engine, remote working, sensor fusion, sharing economy, Skype, supply-chain management, TaskRabbit, trade route, underbanked, urban planning, white picket fence

Thomas Edison15 Innovation and failure, according to Bezos, are ‘inseparable twins’. It is Amazon’s acceptance of failure as a learning experience that sets them apart from other businesses. ‘[E]very single important thing that we have done has taken a lot of risk taking, perseverance, guts, and some of them have worked out, most of them have not’, says Bezos. Let’s be clear, the ones that have worked out – for example, Prime, Amazon Web Services (AWS) and Amazon Echo – have been colossal successes for the company. The 20-year bet and importance of consistency ‘We’re going to be unprofitable for a long time. And that’s our strategy.’ Jeff Bezos, 199716 Wall Street is inherently short-termist, leaving most public companies focused on maximizing profitability and stock performance from quarter to quarter. Amazon does the exact opposite.

With over 100 million members around the globe, Prime has become so much more than a loyalty programme – it’s become a way of life. Amazon has cleverly taken Prime from a scheme that initially centred on delivery perks to an all-encompassing content-streaming, book-lending, photo-storing beast of a membership programme. The result? Higher spend, shopper frequency and retention. We’ll discuss this in greater detail in the next chapter. Amazon Web Services Amazon’s cloud storage service may not directly benefit shoppers, but it has certainly proven to be Amazon’s white knight. Operating margins are consistently in the high single digits and in 2017 the division was responsible for more than 100 per cent of Amazon’s operating profit. Remember Brad Stone’s point about feeding any part of the flywheel to accelerate it? A uniquely profitable division within Amazon means greater opportunity to reinvest in the core retail division.

As referenced in Chapter 2, it is this customer-centric ethos that has served it well as consumers have begun to embrace digitally enabled or enhanced technology shopping tools. It cannot be underestimated, however, how much it helps that Amazon’s core business is founded on technology innovation. Before looking at this innovation, let’s take a step back here, as it wasn’t always this way. Going back to 2002, with necessity truly being the mother of all invention, Amazon Web Services (AWS) was first born of the need for sufficient number-crunching capacity and standardized, automated computing infrastructures on which to run its retail marketplace. Capitalizing on advances in networking, storage, compute power and virtualization, Amazon began reselling its cloud computing capabilities as services in 2006. However, from 2014 to 2015, Amazon saw its stock price fall 20 per cent.


pages: 102 words: 29,596

The Alliance: Managing Talent in the Networked Age by Reid Hoffman, Ben Casnocha, Chris Yeh

Airbnb, Amazon Web Services, centralized clearinghouse, cloud computing, disruptive innovation, Jeff Bezos, Jony Ive, Marc Andreessen, new economy, pre–internet, Silicon Valley, Silicon Valley startup, software as a service, Steve Jobs

And that’s how Lasseter ended up back at Disney as its chief creative officer of Disney Animation Studios.9 Disney’s management hired an entrepreneurial talent like Lasseter, but they treated him as a commodity rather than an ally, and in the process, they lost their chance to develop a multibillion-dollar business. Lasseter would have been happy to develop that business within Disney, but his managers wouldn’t let him. Benjamin Black and Amazon Web Services Amazon didn’t make the same mistake as Disney. Recently, it used the principles of the alliance to generate a new multibillion-dollar business. Amazon has become a leader in the field of cloud computing, thanks to Amazon Web Services (AWS), which allows companies to rent online storage and computing power, rather than buying and operating their own servers. Companies ranging from Fortune 500 giants to one-person start-ups run their businesses on AWS. What most people don’t realize is that the idea for AWS didn’t come from Amazon’s famed entrepreneurial founder and CEO, Jeff Bezos, or even from a member of his executive team, but rather from an “ordinary” employee.


pages: 688 words: 107,867

Python Data Analytics: With Pandas, NumPy, and Matplotlib by Fabio Nelli

Amazon Web Services, centre right, computer vision, Debian, DevOps, Google Earth, Guido van Rossum, Internet of things, optical character recognition, pattern recognition, sentiment analysis, speech recognition, statistical model, web application

Google Trends http://www.google.com/trends/explore Statistics on search volume (as a proportion of total search) for any given term, since 2004. Likebutton http://likebutton.com/ Mines Facebook’s public data—globally and from your own network—to give an overview of what people “Like” at the moment. Miscellaneous and Public Data Sets Amazon Web Services public datasets http://aws.amazon.com/datasets The public data sets on Amazon Web Services provide a centralized repository of public data sets. An interesting dataset is the 1000 Genome Project, an attempt to build the most comprehensive database of human genetic information. Also a NASA database of satellite imagery of Earth is available. DBPedia http://wiki.dbpedia.org Wikipedia contains millions of pieces of data, structured and unstructured, on every subject.

These data sources freely provide information to anyone in need, and they are called open data . Here is a list of some open data available online. You can find a more complete list and details of the open data available online in Appendix B.DataHub ( http://datahub.io/dataset ) World Health Organization ( http://www.who.int/research/en/ ) Data.gov ( http://data.gov ) European Union Open Data Portal ( http://open-data.europa.eu/en/data/ ) Amazon Web Service public datasets ( http://aws.amazon.com/datasets ) Facebook Graph ( http://developers.facebook.com/docs/graph-api ) Healthdata.gov ( http://www.healthdata.gov ) Google Trends ( http://www.google.com/trends/explore ) Google Finance ( https://www.google.com/finance ) Google Books Ngrams ( http://storage.googleapis.com/books/ngrams/books/datasetsv2.html ) Machine Learning Repository ( http://archive.ics.uci.edu/ml/ ) As an idea of open data sources available online, you can look at the LOD cloud diagram ( http://lod-cloud.net ), which displays the connections of the data link among several open data sources currently available on the network (see Figure 1-3).

Publications, Newspapers, and Books New York Times http://developer.nytimes.com/docs Searchable, indexed archive of news articles going back to 1851. Google Books Ngrams http://storage.googleapis.com/books/ngrams/books/datasetsv2.html This source searches and analyzes the full text of any of the millions of books digitized as part of the Google Books project. Musical Data Million Song Data Set http://aws.amazon.com/datasets/6468931156960467 Metadata on over a million songs and pieces of music. Part of Amazon Web Services. Index A Accents, LaTeX Advanced Data aggregation apply() functions transform() function Anaconda Anderson Iris Dataset, see Iris flower dataset Array manipulation joining arrays column_stack() and row_stack() hstack() function vstack() function splitting arrays hsplit() function split() function vsplit() function Artificial intelligence schematization of Artificial neural networks biological networks edges hidden layer input and output layer multi layer perceptron nodes schematization of SLP ( see Single layer perceptron (SLP)) weight B Bar chart 3D error bars horizontal matplotlib multiserial multiseries stacked bar pandas DataFrame representations stacked bar charts x-axis xticks() function Bayesian methods Big Data Bigrams Biological neural networks Blending operation C Caffe2 Chart typology Choropleth maps D3 library geographical representations HTML() function jinja2 JSON and TSV JSON TopoJSON require.config() results US population data source census.gov file TSV, codes HTML() function jinja2.Template pop2014_by_county dataframe population.csv render() function SUMLEV values Classification and regression trees Classification models Climatic data Clustered bar chart IPython Notebook jinja2 render() function Clustering models Collocations Computer vision Concatenation arrays combining concat() function dataframe keys option pivoting hierarchical indexing long to wide format stack() function unstack() function removing Correlation Covariance Cross-validation Cython D Data aggregation apply() functions GroupBy groupby() function operations output of SPLIT-APPLY-COMBINE hierarchical grouping merge() numeric and string values price1 column transform() function Data analysis charts data visualization definition deployment phase information knowledge knowledge domains computer science disciplines fields of application machine learning and artificial intelligence mathematics and statistics problems of open data predictive model process data sources deployment exploration/visualization extraction model validation planning phase predictive modeling preparation problem definition stages purpose of Python and quantitative and qualitative types categorical data numerical data DataFrame pandas definition nested dict operations structure transposition structure Data manipulation aggregation ( see Data aggregation) concatenation discretization and binning group iteration permutation phases of preparation ( see Data preparation) string ( see String manipulation) transformation Data preparation DataFrame merging operation pandas.concat() pandas.DataFrame.combine_first() pandas.merge() procedures of Data structures, operations DataFrame and series flexible arithmetic methods Data transformation drop_duplicates() function mapping adding values axes dict objects replacing values remove duplicates Data visualization adding text axis labels informative label mathematical expression modified of text() function bar chart ( see Bar chart) chart typology contour plot/map data analysis 3D surfaces grid grids, subplots handling date values histogram installation IPython and IPython QtConsole kwargs figures and axes horizontal subplots linewidth plot() function vertical subplots legend chart of legend() function multiseries chart upper-right corner line chart ( see Line chart) matplotlib architecture and NumPy matplotlib library ( see matplotlib library) mplot3d multi-panel plots grids, subplots subplots pie charts axis() function modified chart pandas Dataframe pie() function shadow kwarg plotting window buttons of commands matplotlib and NumPy plt.plot() function properties QtConsole polar chart pyplot module saving, charts HTML file image file source code scatter plot, 3D Decision trees Deep learning artificial ( see Artificial neural networks) artificial intelligence data availability machine learning neural networks and GPUs Python frameworks programming language schematization of TensorFlow ( see TensorFlow) Digits dataset definition digits.images array digit.targets array handwritten digits handwritten number images matplotlib library scikit-learn library Discretization and binning any() function categorical type cut() function describe() function detecting and filtering outliers qcut() std() function value_counts() function Django Dropping E Eclipse (pyDev) Element-wise computation Expression-oriented programming F Financial data Flexible arithmetic methods Fonts, LaTeX G Gradient theory Graphics Processing Unit (GPU) Grouping Group iteration chain of transformations functions on groups mark() function quantiles() function GroupBy object H Handwriting recognition digits dataset handwritten digits, matplotlib library learning and predicting OCR software scikit-learn svc estimator TensorFlow validation set, six digits Health data Hierarchical indexing arrays DataFrame reordering and sorting levels stack() function statistic levels structure two-dimensional structure I IDEs, see Interactive development environments (IDEs) Image analysis concept of convolutions definition edge detection blackandwhite.jpg image black and white system filters function gradients.jpg image gray gradients Laplacian and Sobel filters results source code face detection gradient theory OpenCV ( see Open Source Computer Vision (OpenCV)) operations representation of Indexing functionalities arithmetic and data alignment dropping reindexing Integration Interactive development environments (IDEs) Eclipse (pyDev) Komodo Liclipse NinjaIDE Spyder Sublime Interactive programming language Interfaced programming language Internet of Things (IoT) Interpreted programming language Interpreter characterization Cython Jython PVM PyPy tokenization IPython and IPython QtConsole Jupyter project logo Notebook DataFrames QtConsole shell tools of Iris flower dataset Anderson Iris Dataset IPython QtConsole Iris setosa features length and width, petal matplotlib library PCA decomposition target attribute types of analysis variables J JavaScript D3 Library bar chart CSS definitions data-driven documents HTML importing library IPython Notebooks Jinja2 library pandas dataframe render() function require.config() method web chart creation Jinja2 library Jython K K-nearest neighbors classification decision boundaries 2D scatterplot, sepals predict() function random.permutation() training and testing set L LaTeX accents fonts fractions, binomials, and stacked numbers with IPython Notebook in Markdown Cell in Python 2 Cell with matplotlib radicals subscripts and superscripts symbols arrow symbols big symbols binary operation and relation symbols Delimiters Hebrew lowercase Greek miscellaneous symbols standard function names uppercase Greek Learning phase Liclipse Linear regression Line chart annotate() arrowprops kwarg Cartesian axes color codes data points different series gca() function Greek characters LaTeX expression line and color styles mathematical expressions mathematical function pandas plot() function set_position() function xticks() and yticks() functions Linux distribution LOD cloud diagram Logistic regression M Machine learning (ML) algorithm development process deep learning diabetes dataset features/attributes Iris flower dataset learning problem linear/least square regression coef_ attribute fit() function linear correlation parameters physiological factors and progression of diabetes single physiological factor schematization of supervised learning SVM ( see Support vector machines (SVMs)) training and testing set unsupervised learning Mapping adding values inplace option rename() function renaming, axes replacing values Mathematical expressions with LaTeX, see LaTeX MATLAB matplotlib matplotlib library architecture artist layer backend layer functions and tools layers pylab and pyplot scripting layer (pyplot) artist layer graphical representation hierarchical structure primitive and composite graphical representation LaTeX NumPy Matrix product Merging operation DataFrame dataframe objects index join() function JOIN operation left_index/right_index options left join, right join and outer join left_on and right_on merge() function Meteorological data Adriatic Sea and Po Valley cities Comacchio image of mountainous areas reference standards TheTimeNow website climate data source JSON file Weather Map site IPython Notebook chart representation CSV files DataFrames humidity function linear regression matplotlib library Milan read_csv() function result shape() function SVR method temperature Jupyter Notebook access internal data command line dataframe extraction procedures Ferrara JSON file json.load() function parameters prepare() function RoseWind ( see RoseWind) wind speed Microsoft excel files dataframe data.xls internal module xlrd read_excel() function MongoDB Multi Layer Perceptron (MLP) artificial networks evaluation of experimental data hidden layers IPython session learning phase model definition test phase and accuracy calculation Musical data N Natural Language Toolkit (NLTK) bigrams and collocations common_contexts() function concordance() function corpora downloader tool fileids() function HTML pages, text len() function library macbeth variable Python library request() function selecting words sentimental analysis sents() function similar() function text, network word frequency macbeth variable most_common() function nltk.download() function nltk.FreqDist() function stopwords string() function word search Ndarray array() function data, types dtype (data-type) intrinsic creation type() function NOSE MODULE “Not a Number” data filling, NaN occurrences filtering out NaN values NaN value NumPy library array manipulation ( see Array manipulation) basic operations aggregate functions arithmetic operators increment and decrement operators matrix product ufunc broadcasting compatibility complex cases operator/function BSD conditions and Boolean arrays copies/views of objects data analysis indexing bidimensional array monodimensional ndarray negative index value installation iterating an array ndarray ( see Ndarray) Numarray python language reading and writing array data shape manipulation slicing structured arrays vectorization O Object-oriented programming language OCR, see Optical Character Recognition (OCR) software Open data Open data sources climatic data demographics IPython Notebook matplotlib pandas dataframes pop2014_by_state dataframe pop2014 dataframe United States Census Bureau financial data health data miscellaneous and public data sets musical data political and government data publications, newspapers, and books social data sports data Open Source Computer Vision (OpenCV) deep learning image processing and analysis add() function blackish image blending destroyWindow() method elementary operations imread() method imshow() method load and display merge() method NumPy matrices saving option waitKey() method working process installation MATLAB packages start programming Open-source programming language Optical Character Recognition (OCR) software order() function P Pandas dataframes Pandas data structures DataFrame assigning values deleting column element selection filtering membership value nested dict transposition evaluating values index objects duplicate labels methods NaN values NumPy arrays and existing series operations operations and mathematical functions series assigning values declaration dictionaries filtering values index internal elements, selection operations Pandas library correlation and covariance data structures ( see Pandas data structures) function application and mapping element row/column statistics getting started hierarchical indexing and leveling indexes ( see Indexing functionalities) installation Anaconda development phases Linux module repository, Windows PyPI source testing “Not a Number” data python data analysis sorting and ranking Permutation new_order array np.random.randint() function numpy.random.permutation() function random sampling DataFrame take() function Pickle—python object serialization cPickle frame.pkl pandas library stream of bytes Political and government data pop2014_by_county dataframe pop2014_by_state dataframe pop2014 dataframe Portable programming language PostgreSQL Principal component analysis (PCA) Public data sets PVM, see Python virtual machine (PVM) pyplot module interactive chart Line2D object plotting window show() function PyPy interpreter Python data analysis library deep learning frameworks module OpenCV Python Package Index (PyPI) Python’s world code implementation distributions Anaconda Enthought Canopy Python(x,y) IDEs ( see Interactive development environments (IDEs)) installation interact interpreter ( see Interpreter) IPython ( see IPython) programming language PyPI Python 2 Python 3 running, entire program code SciPy libraries matplotlib NumPy pandas shell source code data structure dictionaries and lists functional programming Hello World index libraries and functions map() function mathematical operations print() function writing python code, indentation Python virtual machine (PVM) PyTorch Q Qualitative analysis Quantitative analysis R R Radial Basis Function (RBF) Radicals, LaTeX Ranking Reading and writing array binary files tabular data Reading and writing data CSV and textual files header option index_col option myCSV_01.csv myCSV_03.csv names option read_csv() function read_table() function .txt extension databases create_engine() function dataframe pandas.io.sql module pgAdmin III PostgreSQL read_sql() function read_sql_query() function read_sql_table() function sqlalchemy sqlite3 DataFrame objects functionalities HDF5 library data structures HDFStore hierarchical data format mydata.h5 HTML files data structures read_html () web_frames web pages web scraping I/O API Tools JSON data books.json frame.json json_normalize() function JSONViewer normalization read_json() and to_json() read_json() function Microsoft excel files NoSQL database insert() function MongoDB pickle—python object serialization RegExp metacharacters read_table() skiprows TXT files nrows and skiprows options portion by portion writing ( see Writing data) XML ( see XML) Regression models Reindexing RoseWind DataFrame hist array polar chart scatter plot representation showRoseWind() function S Scikit-learn library data analysis k-nearest neighbors classification PCA Python module sklearn.svm.SVC supervised learning svm module SciPy libraries matplotlib NumPy pandas Sentimental analysis document_features() function documents list() function movie_reviews negative/positive opinion opinion mining Shape manipulation reshape() function shape attribute transpose() function Single layer perceptron (SLP) accuracy activation function architecture cost optimization data analysis evaluation phase learning phase model definition explicitly implicitly learning phase placeholders tf.add() function tf.nn.softmax() function modules representation testing set test phase and accuracy calculation training sets Social data sort_index() function Sports data SQLite3 stack() function String manipulation built-in methods count() function error message index() and find() join() function replace() function split() function strip() function regular expressions findall() function match() function re.compile() function regex re.split() function split() function Structured arrays dtype option structs/records Subjective interpretations Subscripts and superscripts, LaTeX Supervised learning machine learning scikit-learn Support vector classification (SVC) decision area effect, decision boundary nonlinear number of points, C parameter predict() function regularization support_vectors array training set, decision space Support vector machines (SVMs) decisional space decision boundary Iris Dataset decision boundaries linear decision boundaries polynomial decision boundaries polynomial kernel RBF kernel training set SVC ( see Support vector classification (SVC)) SVR ( see Support vector regression (SVR)) Support vector regression (SVR) curves diabetes dataset linear predictive model test set, data swaplevel() function T TensorFlow data flow graph Google’s framework installation IPython QtConsole MLP ( see Multi Layer Perceptron (MLP)) model and sessions SLP ( see Single layer perceptron (SLP)) tensors operation parameters print() function representations of tf.convert_to_tensor() function tf.ones() method tf.random_normal() function tf.random_uniform() function tf.zeros() method Text analysis techniques definition NLTK ( see Natural Language Toolkit (NLTK)) techniques Theano trigrams() function U, V United States Census Bureau Universal functions (ufunc) Unsupervised learning W Web Scraping Wind speed polar chart representation RoseWind_Speed() function ShowRoseWind() function ShowRoseWind_Speed() function to_csv () function Writing data HTML files myFrame.html to_html() function na_rep option to_csv() function X, Y, Z XML books.xml getchildren() getroot() function lxml.etree tree structure lxml library objectify parse() function tag attribute text attribute


pages: 1,136 words: 73,489

Working in Public: The Making and Maintenance of Open Source Software by Nadia Eghbal

Amazon Web Services, barriers to entry, Benevolent Dictator For Life (BDFL), bitcoin, Clayton Christensen, cloud computing, commoditize, continuous integration, crowdsourcing, cryptocurrency, David Heinemeier Hansson, death of newspapers, Debian, disruptive innovation, en.wikipedia.org, Ethereum, Firefox, Guido van Rossum, Hacker Ethic, Induced demand, informal economy, Jane Jacobs, Jean Tirole, Kevin Kelly, Kickstarter, Kubernetes, Mark Zuckerberg, Menlo Park, Network effects, node package manager, Norbert Wiener, pirate software, pull request, RFC: Request For Comment, Richard Stallman, Ronald Coase, Ruby on Rails, side project, Silicon Valley, Snapchat, social graph, software as a service, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, The Death and Life of Great American Cities, The Nature of the Firm, transaction costs, two-sided market, urban planning, web application, wikimedia commons, Zimmermann PGP

The biggest providers, such as Amazon Web Services and Microsoft Azure, have made hosting costs very cheap or free. That doesn’t mean it can’t get expensive, however. Donald Stufft, who maintains the Python package manager PyPI, estimates that its infrastructure costs two to three million dollars per year in hosting, donated by Fastly.204 As more developers have adopted PyPI, the project’s costs have grown significantly. According to Stufft, in April 2013, PyPI used 11.84GB of bandwidth.205 By April 2019, that figure had increased to 4.5PB.206 On a smaller scale, open source developer Drew DeVault estimates that he spends $380 each month on server hosting for his projects, which he pays for with user donations.207 Werner Vogels, chief technology officer for Amazon and an architect of Amazon Web Services, describes how the marginal cost of physical infrastructure can become significant at scale: Under the covers these services are massive distributed systems that operate on a worldwide scale.

It’s Sorhus’s visibility, and his reputation, that make it possible for him to raise money through sponsorships. Like dependencies, the reputation of a developer is a dynamic property, based on their past and expected future value, viewed through the eyes of others. The practice of paying maintainers to work on open source projects is not new. Donald Stufft, for example, who maintains Python’s packaging tools, was hired by Hewlett Packard Enterprise,253 and then Amazon Web Services, to improve and maintain Python’s packaging.254 A company could just contribute their own employees to a given open source project, and they often do, but some also hire and pay maintainers. The maintainer’s reputation translates into value for the company, whether it’s brand association or having a direct line to someone with influence on the project. What does seem newer, however, is the practice of open source developers raising money from their fans and users, independent not just of a salaried job but of a specific project.


pages: 458 words: 135,206

CTOs at Work by Scott Donaldson, Stanley Siegel, Gary Donaldson

Amazon Web Services, bioinformatics, business intelligence, business process, call centre, centre right, cloud computing, computer vision, connected car, crowdsourcing, data acquisition, distributed generation, domain-specific language, glass ceiling, orbital mechanics / astrodynamics, pattern recognition, Pluto: dwarf planet, QR code, Richard Feynman, Ruby on Rails, shareholder value, Silicon Valley, Skype, smart grid, smart meter, software patent, thinkpad, web application, zero day, zero-sum game

Index 3D technology David Kuttler, 302 Jerry Krill, 83–85 64-bit architectures, 272–274, 281 A acquisitions, William Ballard, 279–280 Adobe, reverse image searching and, 246 AEP (American Electric Power), 128–129 Aguru Images, Craig Miller, 53–54, 56–59 algae, Craig Miller, 69, 76 Alm, Al, 50 Alving, Amy, 1–20 Amazon EC2, 36 Amazon Web Services (AWS), 245 American Electric Power (AEP), 128–129 American Public Media, 163 Amperion, 128 APL (Applied Physics Laboratory) global technical outreach, 90 Jerry Krill, 82 application development, Jeff Tolnar, 145 architecture technical review, 26 Arduinos, Craig Miller, 77 asymmetric warfare, Jerry Krill's comments on, 92 AWS (Amazon Web Services), 245 B Ballard, William, CTO of Gerson Lehrman Group, Inc, 261–283 Beyster, Bob, 51–52 Bilger, Mark, 23 Black, David, 262 Bloore, Paul, 239–260 Bodine, Greg, 30 BPLG (BPL Global), 127–128, 132 breakout engagements, Gerson Lehrman, 265 budgeting, Jeff Tolnar, 146–148 bundling hardware and software, Jeff Tolnar, 135 Burke, Thomas, 177 business development personnel, Gerson Lehrman, 266 C CA Open Space, 39 callable interface, 41 career lessons Darko Hrelic, 206–207, 211 Jan-Erik de Boer, 236 Jerry Krill, 91 Paul Bloore, 253 Wesley Kaplow, 110 career path Amy Alving, 1–6 Darko Hrelic, 205–206 Dmitry Cherches, 181–185 Jan-Erik de Boer, 237 Jeff Tolnar, 128 Marty Garrison, 152 Paul Bloore, 240–242 Tom Loveland, 176–178 Wesley Kaplow, 105–110 CATEX, Craig Miller, 51 Cherches, Dimitri background, 173 of Mind Over Machines, 173–204 ChoicePoint, 158 CIO responsibilities, of William Ballard, 268 CIO role, versus CTO role, 282 CIOs, Jeff Tolnar, 130–131 cloud computing, 35 Amy Alving, 16–17 Craig Miller, 65 David Kuttler, 302 Dmitry Cherches, 186–188, 200 Gerson Lehrman, 277 Jeff Tolnar, 142, 147–148 Marty Garrison, 160, 167 CloudShield, 13 cluster analysis, Craig Miller, 64 CommonCrawl group, 257 communication activities, of William Ballard, 269 communication, importance of, 30 competition, evaluating Amy Alving, 12–13 Darko Hrelic, 211–212, 217 Gerson Lehrman, 265 Jerry Krill, 101–102 Paul Bloore, 244–245 computing cloud computing, 35 Amy Alving, 16–17 Craig Miller, 65 David Kuttler, 302 Dmitry Cherches, 186–188, 200 Gerson Lehrman, 277 Jeff Tolnar, 142, 147–148 Marty Garrison, 160, 167 distributed computing, 242 mobile computing, 39–40 Darko Hrelic, 209–210 impact on Gerson Lehrman, 276 Jerry Krill, 96 Marty Garrison, 168 Paul Bloore, 254 Wesley Kaplow, 123 Comverge, 137 conference calls, ZipDx, 277–278 consulting, by Gerson Lehrman, 265 continuous deployment cycle, at Gerson Lehrman, 281 corporate strategy, Rick Mosca, 191–194 councils, Gerson Lehrman, 264 creativity, role in career path Jerry Krill, 85–87 Paul Bloore, 241 Wesley Kaplow, 116 Crinks, Bert, 115–116 CTO responsibilities, of William Ballard, 267–271, 282–283 Cuomo, Jerry, 25 customizing software, William Ballard's advice regarding, 270–272 cyanobacteria, Craig Miller, 79 cyber security Amy Alving, 18–19 CloudShield, 13 Darko Hrelic, 214–215 Dmitry Cherches, 201 Gerson Lehrman, 276 Springer, 235 Wesley Kaplow, 123 D Dannelly, Doug, 69 DARPA (Defense Advanced Research Projects Agency), Amy Alving, 5–6 Darwin, Charles, 70 data mining, real-time, 281–282 databases Jeff Tolnar, 141 used at Gerson Lehrman, 272 David Kuttler, 296 DAVID system, 165 de Boer, Jan-Erik, 219–237 Debevec, Paul, 57 decision-making, Jeff Tolnar, 133–134 Defense Advanced Research Projects Agency (DARPA), Amy Alving, 5–6 Delman, Debra, 161 Demand Media, 261–262 Department of Energy (DoE), 54–55, 64 deployment cycle, at Gerson Lehrman, 281 DER (distributed energy resources), 139, 142–149 DEs (Distinguished Engineers), 25 development methodology, Paul Bloore, 256–257 development resources, Jeff Tolnar, 139–140 DiData, Craig Miller, 53, 61 digital watermarking, 243 Dimension Data, Craig Miller, 52 Distinguished Engineers (DEs), 25 distributed computing, 242 distributed energy resources (DER), 139, 142–149 Dobson, David, 36 DoE (Department of Energy), 54–55, 64 domain expertise, 36 E economics of consulting with Gerson Lehrman, 265 William Ballard's advice regarding, 270–271, 279 electric co-ops, Craig Miller, 54–55 electrical transformers, Craig Miller, 68 employees, Gerson Lehrman, 265–266 Endhiran movie, Craig Miller, 58–59 energy policy, Craig Miller, 50 engineering offices, Gerson Lehrman, 266 engineers, cost of, 270, 279 ETL (extraction, translation, load), 203 evaluating competition Amy Alving, 12–13 Darko Hrelic, 211–212, 217 Gerson Lehrman, 265 Jerry Krill, 101–102 Paul Bloore, 244–245 expert network space, Gerson Lehrman, 265 expertise, use of, 274–275 external organizations, affiliations of Gerson Lehrman with, 269 extinction, Craig Miller, 78–79 extraction, translation, load (ETL), 203 F FDA, David Kuttler, 300–301 Ferguson, Don, interview with, 21–47 Fernandez, Raul, 52 ferrofluid, 241 First of a Kind project, 23 Folderauer, Ken, 114 fractal architecture, 273–274 free space optics, 84, 95 frequency variations, Craig Miller, 62–63 functional programming, 278 fundraising, Jeff Tolnar, 146 G Garrison, Marty, interview with, 151 Gayal, Amoosh, 24 Gelpin, Tim, 89 G.I.

We now have over two billion images in our index, and a search takes about a second or so. So that in a nutshell is what TinEye is. It was something that I wanted us to do for quite some time. We released it in March of 2008, but I'd wanted to make a web-wide search for a couple of years prior to that, but we didn't actually have the computing capacity to do it. In late 2007, Amazon was developing their AWS, their Amazon Web Services, and cloud computing platform. I was immediately taken by it because I thought, “Aha. Now we can manage to get ahold of the computing capacity we need to launch the search engine.” So we started work on it immediately, and I think it was about four or five months later when we first put TinEye on the web. And, of course, that was about three years ago now. S. Donaldson: Right. So what's an application?

What's the world starting to look like, and how should we position our services so we take advantage of what the world's going to be in five years' time? One future vision I have is currently counter culture. One of the things that I'm kind of known for is that I do worry about costs. I'm kind of referred to as a cheap guy when it comes to spending money on equipment. So we started TinEye completely using cloud-based services on Amazon's Web Services. Pretty soon, though, we realized that we could do it much more cost-effectively on our own equipment. So this means that we moved away from cloud computing and we do everything ourselves in-house. And that's just interesting because it seems to be swimming against the stream as far as what I see when I speak with other technology people. Many start-ups I see are afraid of buying their own equipment.


pages: 474 words: 130,575

Surveillance Valley: The Rise of the Military-Digital Complex by Yasha Levine

23andMe, activist fund / activist shareholder / activist investor, Airbnb, AltaVista, Amazon Web Services, Anne Wojcicki, anti-communist, Apple's 1984 Super Bowl advert, bitcoin, borderless world, British Empire, call centre, Chelsea Manning, cloud computing, collaborative editing, colonial rule, computer age, computerized markets, corporate governance, crowdsourcing, cryptocurrency, digital map, don't be evil, Donald Trump, Douglas Engelbart, Douglas Engelbart, drone strike, Edward Snowden, El Camino Real, Electric Kool-Aid Acid Test, Elon Musk, fault tolerance, George Gilder, ghettoisation, global village, Google Chrome, Google Earth, Google Hangouts, Howard Zinn, hypertext link, IBM and the Holocaust, index card, Jacob Appelbaum, Jeff Bezos, jimmy wales, John Markoff, John von Neumann, Julian Assange, Kevin Kelly, Kickstarter, life extension, Lyft, Mark Zuckerberg, market bubble, Menlo Park, Mitch Kapor, natural language processing, Network effects, new economy, Norbert Wiener, packet switching, PageRank, Paul Buchheit, peer-to-peer, Peter Thiel, Philip Mirowski, plutocrats, Plutocrats, private military company, RAND corporation, Ronald Reagan, Ross Ulbricht, Satoshi Nakamoto, self-driving car, sentiment analysis, shareholder value, side project, Silicon Valley, Silicon Valley startup, Skype, slashdot, Snapchat, speech recognition, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, Telecommunications Act of 1996, telepresence, telepresence robot, The Bell Curve by Richard Herrnstein and Charles Murray, The Hackers Conference, uber lyft, Whole Earth Catalog, Whole Earth Review, WikiLeaks

The Obama campaign used AWS, and over 18 months built 200 applications. On election day they built a call center, they built an elaborate database to know where their volunteers were, know the neighborhoods where people appeared not to have voted, so they could go knock on doors and get out the vote,” Andy Jassy, head of Amazon Web Services, told All Things Digital. “Nine Questions for Andy Jassy, Head of Amazon Web Services,” All Things Digital, November 8, 2013, https://web.archive.org/web/20170528161820/http://allthingsd.com/20131108 /nine-questions-for-andy-jassy-head-of-amazon-web-services/comment-page-1/. 136. Adi Robertson, “Jeff Bezos’ Blue Origin Partners with Boeing and Lockheed Martin to Reduce Dependence on Russian Rockets,” The Verge, September 17, 2014, https://www.theverge.com/2014/9/17/6328961/jeff-bezos-blue-origin-partners-with-united-launch-alliance-for-new-rocket. 137.

Brad Stone, The Everything Store: Jeff Bezos and the Age of Amazon (Boston: Little, Brown, 2013); Jennifer Wills, “7 Ways Amazon Uses Big Data to Stalk You,” Investopedia, September 7, 2016, http://www.investopedia.com/articles /insights/090716/7-ways-amazon-uses-big-data-stalk-you-amzn.asp; Greg Bensinger, “Amazon Wants to Ship Your Package Before You Buy It,” Wall Street Journal, January 17, 2014. 83. Hal Bernton and Susan Kelleher, “Amazon Warehouse Jobs Push Workers to Physical Limit,” Seattle Times, April 3, 2012. 84. Dan Frommer, “Amazon Web Services Is Approaching a $10 Billion-a-Year Business,” Recode, April 28, 2016, https://www.recode.net/2016/4/28/11586526 /aws-cloud-revenue-growth. 85. Went on a buck-hunting trip with your grandson? A Republican candidate could target you for gun-rights ads. Belong to an evangelical Bible study group? Maybe the candidate will show you something about fighting abortion instead. Facebook allowed politicians to target voters with laser precision on the basis of information they had never been able to collect before.


pages: 444 words: 127,259

Super Pumped: The Battle for Uber by Mike Isaac

"side hustle", activist fund / activist shareholder / activist investor, Airbnb, Albert Einstein, always be closing, Amazon Web Services, Andy Kessler, autonomous vehicles, Ayatollah Khomeini, barriers to entry, Bay Area Rapid Transit, Burning Man, call centre, Chris Urmson, Chuck Templeton: OpenTable:, citizen journalism, Clayton Christensen, cloud computing, corporate governance, creative destruction, don't be evil, Donald Trump, Elon Musk, family office, gig economy, Google Glasses, Google X / Alphabet X, high net worth, Jeff Bezos, John Markoff, Kickstarter, Lyft, Marc Andreessen, Mark Zuckerberg, mass immigration, Menlo Park, Mitch Kapor, money market fund, moral hazard, move fast and break things, move fast and break things, Network effects, new economy, off grid, peer-to-peer, pets.com, Richard Florida, ride hailing / ride sharing, Sand Hill Road, self-driving car, shareholder value, side project, Silicon Valley, Silicon Valley startup, skunkworks, Snapchat, software as a service, software is eating the world, South China Sea, South of Market, San Francisco, sovereign wealth fund, special economic zone, Steve Jobs, TaskRabbit, the payments system, Tim Cook: Apple, Travis Kalanick, Uber and Lyft, Uber for X, uber lyft, ubercab, union organizing, upwardly mobile, Y Combinator

First, By 2008, more than 75 percent of American households owned computers, and unlike the 1990s and early 2000s, this mass population had access to broadband; more than half of American adults in 2008 purchased a high-speed internet connection for the home. As more and more people connected online, demand for new, internet-enabled services grew by the day. Second, the hurdles for entrepreneurs who wanted to launch a company were lowering quickly. Amazon Web Services, or AWS, changed the startup game entirely. Amazon started AWS in 2002 as an engineering side project; it would grow to become one of its most successful innovations in Amazon history. Amazon Web Services powers cloud computing services for coders and entrepreneurs who can’t afford to build their own infrastructure or server farms on their own. If a startup is a house, AWS is the electric company, the foundation and the plumbing combined. It keeps the business up and running while the company founders can spend their time focusing on more important things like, say, getting people to come to their house in the first place.

Places like Best Buy, FuncoLand, and Babbage’s had aisles stocked like a grocery store, stuffed with rows of boxes of PC and Mac programs. The App Store changed the model for software development entirely. All a programmer needed was an idea and facility with Apple’s mobile software code. With those two components, anyone could build and distribute their own apps and market them to millions of people instantly. Spin up a server on Amazon Web Services, blast out some code, and submit your app to Apple for review, and your work could be up and running in days. For people opening the App Store at home, it was like walking the aisles of their local Best Buy. Unfettered access to millions of games and programs on their iPhone required little more than a Wi-Fi connection and a few extra bucks. Coders across the world looked at the App Store with giant, flashing dollar signs in their eyes.

INDEX Page numbers followed by n refer to footnotes. 510 Systems, 108 ABC, 131 Abu Dhabi Investment Authority, 317 Abyzov, Ilya, 90 Accel, 40 Adobe, 27, 39 Advanced Micro Devices, 67 Advanced Technologies Group, 184 Agenda, 67 AIG, 33 Airbnb, xiii, 6, 65, 224 Akamai Technologies, 29, 31, 32 Albarrán, Tammy, 224, 226, 271–72, 275 Alexander, Eric, 260–61, 262, 337 Alfonso, Melene, 16 Alipay, 148 Allen & Company, 319, 320 Alltel, 100 Alphabet, 232. See also Google Alsup, William, 255–56 Amazon, 4, 11–14, 27, 35, 68–69, 92n, 111, 115, 154, 195, 231n Amazon Web Services (AWS), 34–35, 39 American Civil Liberties Union, 211 Andreessen, Marc, 69n, 75 Andreessen Horowitz, 40, 184 Android, 156, 159 AOL, 31, 69n, 229, 230 APAC, 260 Apple, 4, 36, 38–40, 59, 132, 165, 195, 233, 235, 245, 317. See also specific products the Apple problem, 153–64 App Store, 37, 39–40, 59, 155, 157–63, 235, 245 Aquafina, 60 Arab Spring, 199 Arlene Schnitzer Concert Hall, xv Arrington, Michael, 55, 58 Arro, 113 Asia, 139, 140–52, 187, 259, 260.


Martin Kleppmann-Designing Data-Intensive Applications. The Big Ideas Behind Reliable, Scalable and Maintainable Systems-O’Reilly (2017) by Unknown

active measures, Amazon Web Services, bitcoin, blockchain, business intelligence, business process, c2.com, cloud computing, collaborative editing, commoditize, conceptual framework, cryptocurrency, database schema, DevOps, distributed ledger, Donald Knuth, Edward Snowden, Ethereum, ethereum blockchain, fault tolerance, finite state, Flash crash, full text search, general-purpose programming language, informal economy, information retrieval, Internet of things, iterative process, John von Neumann, Kubernetes, loose coupling, Marc Andreessen, microservices, natural language processing, Network effects, packet switching, peer-to-peer, performance metric, place-making, premature optimization, recommendation engine, Richard Feynman, self-driving car, semantic web, Shoshana Zuboff, social graph, social web, software as a service, software is eating the world, sorting algorithm, source of truth, SPARQL, speech recognition, statistical model, undersea cable, web application, WebSocket, wikimedia commons

. • Free and open source software has become very successful and is now preferred to commercial or bespoke in-house software in many environments. • CPU clock speeds are barely increasing, but multi-core processors are standard, and networks are getting faster. This means parallelism is only going to increase. • Even if you work on a small team, you can now build systems that are distributed across many machines and even multiple geographic regions, thanks to infra‐ structure as a service (IaaS) such as Amazon Web Services. • Many services are now expected to be highly available; extended downtime due to outages or maintenance is becoming increasingly unacceptable. Data-intensive applications are pushing the boundaries of what is possible by making use of these technological developments. We call an application data-intensive if data is its primary challenge—the quantity of data, the complexity of data, or the speed at which it is changing—as opposed to compute-intensive, where CPU cycles are the bottleneck.

As long as you can restore a backup onto a new machine fairly quickly, the downtime in case of failure is not catastrophic in most applications. Thus, multi-machine redundancy was only required by a small number of applications for which high availability was absolutely essential. However, as data volumes and applications’ computing demands have increased, more applications have begun using larger numbers of machines, which proportion‐ ally increases the rate of hardware faults. Moreover, in some cloud platforms such as Amazon Web Services (AWS) it is fairly common for virtual machine instances to become unavailable without warning [7], as the platforms are designed to prioritize flexibility and elasticityi over single-machine reliability. Hence there is a move toward systems that can tolerate the loss of entire machines, by using software fault-tolerance techniques in preference or in addition to hardware redundancy. Such systems also have operational advantages: a single-server system requires planned downtime if you need to reboot the machine (to apply operating system security patches, for example), whereas a system that can tolerate machine failure can be patched one node at a time, without downtime of the entire system (a rolling upgrade; see Chapter 4).

.: “Availability in Glob‐ ally Distributed Storage Systems,” at 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI), October 2010. [6] Brian Beach: “Hard Drive Reliability Update – Sep 2014,” backblaze.com, Septem‐ ber 23, 2014. [7] Laurie Voss: “AWS: The Good, the Bad and the Ugly,” blog.awe.sm, December 18, 2012. Summary | 23 [8] Haryadi S. Gunawi, Mingzhe Hao, Tanakorn Leesatapornwongsa, et al.: “What Bugs Live in the Cloud?,” at 5th ACM Symposium on Cloud Computing (SoCC), November 2014. doi:10.1145/2670979.2670986 [9] Nelson Minar: “Leap Second Crashes Half the Internet,” somebits.com, July 3, 2012. [10] Amazon Web Services: “Summary of the Amazon EC2 and Amazon RDS Ser‐ vice Disruption in the US East Region,” aws.amazon.com, April 29, 2011. [11] Richard I. Cook: “How Complex Systems Fail,” Cognitive Technologies Labora‐ tory, April 2000. [12] Jay Kreps: “Getting Real About Distributed System Reliability,” blog.empathy‐ box.com, March 19, 2012. [13] David Oppenheimer, Archana Ganapathi, and David A. Patterson: “Why Do Internet Services Fail, and What Can Be Done About It?


pages: 1,237 words: 227,370

Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems by Martin Kleppmann

active measures, Amazon Web Services, bitcoin, blockchain, business intelligence, business process, c2.com, cloud computing, collaborative editing, commoditize, conceptual framework, cryptocurrency, database schema, DevOps, distributed ledger, Donald Knuth, Edward Snowden, Ethereum, ethereum blockchain, fault tolerance, finite state, Flash crash, full text search, general-purpose programming language, informal economy, information retrieval, Infrastructure as a Service, Internet of things, iterative process, John von Neumann, Kubernetes, loose coupling, Marc Andreessen, microservices, natural language processing, Network effects, packet switching, peer-to-peer, performance metric, place-making, premature optimization, recommendation engine, Richard Feynman, self-driving car, semantic web, Shoshana Zuboff, social graph, social web, software as a service, software is eating the world, sorting algorithm, source of truth, SPARQL, speech recognition, statistical model, undersea cable, web application, WebSocket, wikimedia commons

Free and open source software has become very successful and is now preferred to commercial or bespoke in-house software in many environments. CPU clock speeds are barely increasing, but multi-core processors are standard, and networks are getting faster. This means parallelism is only going to increase. Even if you work on a small team, you can now build systems that are distributed across many machines and even multiple geographic regions, thanks to infrastructure as a service (IaaS) such as Amazon Web Services. Many services are now expected to be highly available; extended downtime due to outages or maintenance is becoming increasingly unacceptable. Data-intensive applications are pushing the boundaries of what is possible by making use of these technological developments. We call an application data-intensive if data is its primary challenge—the quantity of data, the complexity of data, or the speed at which it is changing—as opposed to compute-intensive, where CPU cycles are the bottleneck.

As long as you can restore a backup onto a new machine fairly quickly, the downtime in case of failure is not catastrophic in most applications. Thus, multi-machine redundancy was only required by a small number of applications for which high availability was absolutely essential. However, as data volumes and applications’ computing demands have increased, more applications have begun using larger numbers of machines, which proportionally increases the rate of hardware faults. Moreover, in some cloud platforms such as Amazon Web Services (AWS) it is fairly common for virtual machine instances to become unavailable without warning [7], as the platforms are designed to prioritize flexibility and elasticityi over single-machine reliability. Hence there is a move toward systems that can tolerate the loss of entire machines, by using software fault-tolerance techniques in preference or in addition to hardware redundancy. Such systems also have operational advantages: a single-server system requires planned downtime if you need to reboot the machine (to apply operating system security patches, for example), whereas a system that can tolerate machine failure can be patched one node at a time, without downtime of the entire system (a rolling upgrade; see Chapter 4).

[6] Brian Beach: “Hard Drive Reliability Update – Sep 2014,” backblaze.com, September 23, 2014. [7] Laurie Voss: “AWS: The Good, the Bad and the Ugly,” blog.awe.sm, December 18, 2012. [8] Haryadi S. Gunawi, Mingzhe Hao, Tanakorn Leesatapornwongsa, et al.: “What Bugs Live in the Cloud?,” at 5th ACM Symposium on Cloud Computing (SoCC), November 2014. doi:10.1145/2670979.2670986 [9] Nelson Minar: “Leap Second Crashes Half the Internet,” somebits.com, July 3, 2012. [10] Amazon Web Services: “Summary of the Amazon EC2 and Amazon RDS Service Disruption in the US East Region,” aws.amazon.com, April 29, 2011. [11] Richard I. Cook: “How Complex Systems Fail,” Cognitive Technologies Laboratory, April 2000. [12] Jay Kreps: “Getting Real About Distributed System Reliability,” blog.empathybox.com, March 19, 2012. [13] David Oppenheimer, Archana Ganapathi, and David A. Patterson: “Why Do Internet Services Fail, and What Can Be Done About It?


pages: 368 words: 96,825

Bold: How to Go Big, Create Wealth and Impact the World by Peter H. Diamandis, Steven Kotler

3D printing, additive manufacturing, Airbnb, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, Charles Lindbergh, cloud computing, creative destruction, crowdsourcing, Daniel Kahneman / Amos Tversky, dematerialisation, deskilling, disruptive innovation, Elon Musk, en.wikipedia.org, Exxon Valdez, fear of failure, Firefox, Galaxy Zoo, Google Glasses, Google Hangouts, gravity well, ImageNet competition, industrial robot, Internet of things, Jeff Bezos, John Harrison: Longitude, John Markoff, Jono Bacon, Just-in-time delivery, Kickstarter, Kodak vs Instagram, Law of Accelerating Returns, Lean Startup, life extension, loss aversion, Louis Pasteur, low earth orbit, Mahatma Gandhi, Marc Andreessen, Mark Zuckerberg, Mars Rover, meta analysis, meta-analysis, microbiome, minimum viable product, move fast and break things, Narrative Science, Netflix Prize, Network effects, Oculus Rift, optical character recognition, packet switching, PageRank, pattern recognition, performance metric, Peter H. Diamandis: Planetary Resources, Peter Thiel, pre–internet, Ray Kurzweil, recommendation engine, Richard Feynman, ride hailing / ride sharing, risk tolerance, rolodex, self-driving car, sentiment analysis, shareholder value, Silicon Valley, Silicon Valley startup, skunkworks, Skype, smart grid, stem cell, Stephen Hawking, Steve Jobs, Steven Levy, Stewart Brand, superconnector, technoutopianism, telepresence, telepresence robot, Turing test, urban renewal, web application, X Prize, Y Combinator, zero-sum game

• We will share our strategic thought processes with you when we make bold choices (to the extent competitive pressures allow), so that you may evaluate for yourselves whether we are making rational long-term leadership investments. • We will balance our focus on growth with emphasis on long-term profitability and capital management. At this stage, we choose to prioritize growth because we believe that scale is central to achieving the potential of our business model. This letter is often held up as the encapsulation of Bezos’s view on the subject, but personally, I think an answer he gave to an Amazon Web Services Live audience in 201227 was far more revealing: “What’s going to change in the next ten years?” And that is a very interesting question; it’s a very common one. I almost never get the question: “What’s not going to change in the next ten years?” And I submit to you that that second question is actually the more important of the two—because you can build a business strategy around the things that are stable in time. . . .

But it’s the combination of long-term thinking and customer-centrism that has helped Amazon extend their reach far beyond books. Bezos has ventured into music, movies, toys, electronics, automotive parts, and well, just about everything. They have also continued to surround their original market, moving from books into ebooks and ebook readers (with Kindle), and most recently, publishing itself. Meanwhile, Amazon Web Services—their cloud business—has become a beast in its own right (worth nearly $3 billion, according to a November 2013 Business Insider analysis).28 As Morgan Stanley analyst Scott Devitt told the New York Times:29 “Amazon is marching to a different drumbeat, which is long term. Are they doing the right thing? Absolutely. Amazon is growing at twice the rate of e-commerce as a whole, which is growing five times faster than retail over all.

Italic numbers refer to charts/graphs Aabar, 127 Abundance (Diamandis and Kotler), xi–xii, xv, 34, 54, 136, 137, 146, 162, 274 AbundanceHub.com, 158, 162, 210, 277 Abundance360Summit, 278 Academy of Achievement, 129 activists, xiii, 180 in crowdfunding campaigns, 201–3, 212, 230 additive manufacturing, 30, 31, 33, 41 AdhereTech, 47 AdSense, 139 Advanced Research Projects Agency Network (ARPANET), 27 advertising, 241, 242 in crowdfunding campaigns, 212–13 crowdsourcing platforms for, 151, 152–54, 158 advocates, in crowdfunding campaigns, 200–201, 205 AdWords, 241 aerospace industry, 112, 117, 133 skunk methodology used in, 71–73, 75 3–D printing and, 34, 35–37 see also space exploration affiliate marketing, 199–200, 205 Ahn, Luis von, 154, 155–56 Airbnb, 20, 21, 66 Airbus, 249 airlines, 43, 124, 125, 126, 127, 260 AI XPRIZE, 54 algorithms, 43, 51, 52, 66, 85, 220, 227 crowdsourcing projects and, 158, 159, 160, 161, 227 machine-learning, 54–55, 55, 58 Netflix Prize for, 254–56 PageRank, 135 Align Technology, 34–35 Amazon, 50, 76, 97, 128, 129–34, 157, 195, 254 drone proposal of, 61, 133–34 Amazon Web Services, 131, 132 America Online (AOL), 76, 143 Anderson, Chris, 10–11, 54, 123, 224, 229, 242 Anderson, Eric, 95, 96, 97, 98, 99, 100, 179, 202, 203–4 Andraka, Jack, 65 Andreessen, Marc, 27, 33 Android, 16, 135, 176 AngelList, 172, 173–74 Anheuser-Busch, 145 Ansari XPRIZE, 76, 96, 115, 127, 134, 246, 248–49, 253, 260, 261, 262, 263, 264, 265, 266, 267, 268 anti-aging projects, 66, 81, 136, 139 Apollo Program, 96, 100, 118, 139 Appert, Nicolas, 245 Appirio, 227–28 Apple, 18, 28, 62, 72, 111, 128 applications (apps), 13, 13, 15, 16, 28, 45, 47, 150, 158, 176 Arduino, 43 ARKYD Space Telescope campaign, 172, 174–75, 179–80, 186, 187, 188, 193, 195, 207, 209, 242 early donor engagement in, 203–5 hype created in, 205 launch of, 200, 208 recruiting of activists in, 201–3, 212 ”space selfie” reward offered in, 180, 189–90, 196, 208 Arnaout, Ramy, 227 artificial intelligence (AI), x, 22, 24, 41, 44, 52–59, 61, 62, 63, 66, 81, 135, 146, 160, 162, 216, 228, 275, 276, 295n crowdsourcing projects and, 167, 295n entrepreneurial opportunities in, 54, 56–59 Google’s development of, 24, 53, 58, 81, 138–39 Association of Space Explorers, 102 asteroids, ix–x, 180, 228–29 mining of, 95–96, 97–99, 107, 109, 179, 221, 276 Asteroid Zoo, 228 astronomy, 219–21, 228, 247, 267 Autodesk, 48–49, 51, 63, 65 automation, 47–48, 56 automobile industry, 29, 222–23 3–D printing in, 32 see also Local Motors; Tesla Motors autonomous cars, 43–44, 44, 48, 62, 66, 135, 136, 137, 262 autonomy, 79, 80, 85, 87, 92 Babson School of Business, 14 BackRub (research project), 135 Bacon, Jono, 237 Bad Girl Ventures, 19 bake-offs, in incentive competitions, 264 Barnett, Chance, 173 Barrie, Matt, 149–50, 158, 163, 165, 166, 167, 207 Barry, Dan, 35, 61, 62 Bass, Carl, 48–49, 50, 51 Baxter (robot), 60–61 Beland, Francis, 250 Bennett, Jim, 255 Berns, Gregory, 108 Beth Israel Deaconess Medical Center (Boston), 227 Better Blocks, 240–41 Bezos, Jeff, xiii, 73, 97, 115, 126, 128–34, 138, 139, 167 on risk management, 76–77 thinking-at-scale strategies of, 128, 129, 130–33 Bezos, Mark, 128 Bianchini, Gina, 217, 219, 224, 233 Big Think, 49, 121 biotechnology, 63–65 see also genomics; synthetic biology BlackBerry, 176 Blakey, Marion, 110 Blastar (video game), 117 blogs, in crowdfunding campaigns, 177, 205, 206 blue ellipticals, 219–20 Blue Origin, 97, 133 Boeing, 127, 249 Boston Dynamics, 61 Boston Globe, 227 Brand, Stewart, 26 Branson, Richard, xiii, 73, 84, 86, 99, 100, 111–12, 115, 123–28, 138, 139 space tourism projects of, 96–97, 115, 125, 127 thinking-at-scale strategies of, 125–27 Briggs, William, 47 Brin, Sergey, 81, 128, 135 British Admiralty, 267 British Airways, 124, 125, 126 British Medical Journal, 109 British Petroleum (BP), 250, 251 Brooks, Rodney A., 60 Brown, Dan, 152 brute force, 51 Burchard, Brendon, 210 Business Insider, 132 Business World, 144 buzz marketing, 240–41 Bye, Stephen, 45 Calacanis, Jason, 139 Calico, 139 Callaghan, Jon, 62 Caltech, 27 camel racing, 59–60 cameras, 3–4, 152 see also digital cameras Cameron, James, 250 campaign managers, in crowdfunding, 192, 194 Canadian Space Agency (CSA), 102 Cane, Daniel, 57 Capp, Al, 71, 72 CAPTCHA, 154, 155, 167, 295n CastingWords, 145 celebrity (the face): in crowdfunding campaigns, 192, 198, 207 in incentive competitions, 273 cell phones, 49, 135, 163 see also smartphones CFM International, 34 challenge/skills ratio, 91 “charge-coupled device” (CCD), 4–5 Chen, Michael, 35, 36, 37 China, 17, 18, 62 Chinese National Space Administration, 102 Chrome, 135, 138 Chung, Anshe, 144 Cinematch, 254, 255 Cisco, 46 Clarke, Arthur C., 52, 53, 100 cloud services, 39, 45, 50, 51, 56, 57, 63, 65, 66, 132, 216, 227–28 CNN, 48 cognitive biases, 246 cognitive surplus, 215 CoheroHealth, 47 Colgate Palmolive, 154 Comedy Central, 95 communities, online, 22, 182, 215–42, 243 building member base in early days of, 233–34 case studies of, 219–28 collaborative structures of, 217, 227, 228, 236, 237, 255 contests and competitions in building of, 224, 225–27, 232, 237, 240; see also incentive competitions DIY, see DIY communities driving growth in, 239–41 engagement strategies in, 224, 227, 235, 236–38, 239, 241 exponential, see exponential communities Law of Niches in, 221, 223, 228, 231 managing of, 238–39 monetization of, 241–42 passion as important in, 224, 225, 228, 231, 258 rate of innovation in, 216, 219, 224, 225, 228, 233, 237 rating systems in, 226, 232, 236–37 reputation economics in building of, 217–19, 230, 232, 236–37 self-organizing structures of, 217, 237 see also crowdfunding, crowdfunding campaigns; crowdsourcing Compaq, 117 computers, x, 7, 26, 72, 76, 135 see also artificial intelligence (AI); supercomputers Comsat, 102 constraints, power of, 248–49, 259 contract research and manufacturing services (CRAMS), 65 Coolest Cooler campaign, 210–13 corporate sponsorship, 246, 246 Cotichini, Christian, 257 Cotteleer, Mark, 33 Coulson, Simon, 150 Craigslist, 11, 257 creative assets, crowdsourcing of, 158 Creative Commons license, 224 Credit Suisse, 56 Cretaceous Period, ix CrossFit, 229 Crowdfunder, 172, 173, 175 crowdfunding, crowdfunding campaigns, xiii, 22, 103, 144–45, 147–48, 167, 169–213, 216, 242, 243, 247, 258, 270 advertising in, 212–13 building perfect team for, 191–94 building your audience in, 199–203 case studies in, 174–80 celebrity face of, 192, 198, 207 choosing idea for, 184–85 costs in, 195 data-driven decision making in, 207–10, 213 emergence of, 170–71, 170 engagement strategies in, 203–6, 207 feedback in, 176, 180, 182, 185, 190, 199, 200, 202, 209–10 fundraising targets in, 185–87, 191 global focus in, 209 how-to guide to, 181–213 launching with super-credibility in, 190, 199, 203, 204 length and schedule for, 187–89 pitch videos in, 177, 180, 192, 193, 195, 198–99, 203, 212 planning, materials, and resources in, 194–95 promotions and contests in, 207 reward-based, see reward-based crowdfunding setting rewards in, 189–91, 189 seven benefits of, 181–83 telling meaningful story in, 195–98 types of, 172–75 week-by-week execution plan for, 206–7 see also specific crowdfunding campaigns Crowdsortium, 162–63 crowdsourcing, xiii, 18, 22, 57, 85, 103, 143–67, 193, 223, 237, 240–41, 243, 245, 256, 275 in advertising, 151, 152–54, 158 AI as potential threat to, 167, 295n in automotive production, 223–24, 238 best practices of, 163–67 building communities for, see communities, online clear roles and communication in, 165–66 collaboration in, 144, 165–66, 167, 217, 227, 228, 236, 237, 255, 260–61 competitions and, 148, 152–54, 159, 160, 223, 224, 226–27, 232, 237, 240, 259; see also incentive competitions of creative and operational assets, 158–60 definition of, 144 in designing incentive competitions, 257–58 dual-use, 154–56 Freelancer.com case study in, 149–51, 158 growing interconnectedness and, 146–47, 147 incentive competitions and, see incentive competitions industry websites on, 162–63 of micro- vs. macrotasks, 156–58 most common uses for, 156–67 in product development, 18, 19, 223–25, 226–27 in retail and consumer products industry, 159–60 in scientific research, 145–46, 220–21, 227, 228–29 in software development, 144, 159, 161, 226–27, 236 of testing and discovery insights, 160–62 traffic data garnered by, 47 Crowdsourcing.org, 162 Csikszentmihalyi, Mihaly, 89, 92 Cube, 32 CubeSats, 36–37 Culver, Irv, 72 Cummins Engine, 222 Curiosity rover, 99 customer-centric business, 84, 116, 126, 128, 130, 131–32, 133, 138 Daily Show, 95 DARPA Grand Challenge, 262 Dartmouth Summer Research Project, 59 data mining, 42–44, 47–48, 256 AI’s role in, 55–59 behavior tracking and, 47 see also information data sets, preparing of, 164 Da Vinci Code, The (Brown), 152 debt funding, 172, 173, 174 deceptive phase, exponential, 8, 8, 9, 10, 24, 25–26, 29, 41 of AI, 59 in robotics, 60 of 3–D printing, 30, 31 Deep Learning (algorithm), 58, 59 Deepwater Horizon oil rig, 250 Defense Department, US, 71, 72 DeHart, Jacob, 143, 144 DeJulio, James, 151–52, 153, 166 Dell, 50 Deloitte Center for the Edge, 106 Deloitte Consulting, 33, 39, 159, 160, 245, 274 Deloitte University Press, 56 dematerialization, exponential, 8, 8, 10, 11–13, 14, 15, 20–21, 66 democratization, exponential, xii, 8, 10, 13–15, 21, 33, 59, 276 in bioengineering, 64–65 infinite computing and, 51–52 demonetization, exponential, 8, 8, 10–11, 14, 15, 138, 163, 167, 223 in bioengineering, 64–65 infinite computing and, 52 D.


pages: 348 words: 97,277

The Truth Machine: The Blockchain and the Future of Everything by Paul Vigna, Michael J. Casey

3D printing, additive manufacturing, Airbnb, altcoin, Amazon Web Services, barriers to entry, basic income, Berlin Wall, Bernie Madoff, bitcoin, blockchain, blood diamonds, Blythe Masters, business process, buy and hold, carbon footprint, cashless society, cloud computing, computer age, computerized trading, conceptual framework, Credit Default Swap, crowdsourcing, cryptocurrency, cyber-physical system, dematerialisation, disintermediation, distributed ledger, Donald Trump, double entry bookkeeping, Edward Snowden, Elon Musk, Ethereum, ethereum blockchain, failed state, fault tolerance, fiat currency, financial innovation, financial intermediation, global supply chain, Hernando de Soto, hive mind, informal economy, intangible asset, Internet of things, Joi Ito, Kickstarter, linked data, litecoin, longitudinal study, Lyft, M-Pesa, Marc Andreessen, market clearing, mobile money, money: store of value / unit of account / medium of exchange, Network effects, off grid, pets.com, prediction markets, pre–internet, price mechanism, profit maximization, profit motive, ransomware, rent-seeking, RFID, ride hailing / ride sharing, Ross Ulbricht, Satoshi Nakamoto, self-driving car, sharing economy, Silicon Valley, smart contracts, smart meter, Snapchat, social web, software is eating the world, supply-chain management, Ted Nelson, the market place, too big to fail, trade route, transaction costs, Travis Kalanick, Turing complete, Uber and Lyft, uber lyft, unbanked and underbanked, underbanked, universal basic income, web of trust, zero-sum game

If you are already uncomfortable with what Google, Amazon, Facebook, and Apple know about you, think about it in the context of a centralized IoT. Having transactions pass through a small number of behemoth companies would not only be an inefficient way to route data and require regulatory constraints to the system, but it could also create an Orwellian level of control. Do we really want Amazon Web Services or any other big cloud service provider controlling all that valuable data? Not only would that company gain an unprecedented, privileged window onto the entire world of material things and human activity, but it would, in effect, put those centrally controlled companies in charge of what will be billions of machine-to-machine transactions of tokens and digital currencies. That would give a new meaning to the phrase “too big to fail.”

The concept under development by key players such as ConsenSys, Blockstack, and Microsoft is not to directly store the data certifying someone’s or some entity’s identity in transactions on a blockchain. That would overwhelm a distributed ledger’s limited storage capacity very quickly—certainly that of Bitcoin. Rather, the data will reside off-chain, wherever the person or institution chooses to store it: locally on their own computer, smartphone, or other device; with a cloud computing service from IBM, Microsoft, or Amazon Web Service. All those options, of course, require some level of trust in the provider. So it’s interesting that some of the emerging decentralized systems for storing data over the Internet, such as Maidsafe, Storj, IPFS (the Interplanetary File System), or Sia, are also being touted as personal data management tools for identity purposes. Those hosting systems aren’t controlled by a company. Still, there is some vital information that must be stored in a blockchain environment.

There are blockchain-based offerings looking to disintermediate the business of outsourced storage and computing, for example, to break the expensive, wasteful, and environmentally harmful dominance of corporate-owned data centers. With names such as Storj, Sia, and Maidsafe, these new platforms reward you with tokens if you offer up your spare hard-drive space to other computer users in a global network of users. You could say these “cloud” services are much truer to that name than those of Amazon Web Services, Google, Dropbox, IBM, Oracle, Microsoft, and Apple, the providers with which most people associate that word. But even bigger changes are being considered, including projects to entirely re-architect the Web itself. There’s Solid, which stands for Social Linked Data, a new protocol for data storage that puts data back in the hands of the people to whom it belongs. The core idea is that we will store our data in Pods (Personalized Online Data Stores) and distribute it to applications via permissions we control.


pages: 209 words: 53,175

The Psychology of Money: Timeless Lessons on Wealth, Greed, and Happiness by Morgan Housel

"side hustle", airport security, Amazon Web Services, Bernie Madoff, business cycle, computer age, coronavirus, discounted cash flows, diversification, diversified portfolio, Donald Trump, financial independence, Hans Rosling, Hyman Minsky, income inequality, index fund, invisible hand, Isaac Newton, Jeff Bezos, Joseph Schumpeter, knowledge worker, labor-force participation, Long Term Capital Management, margin call, Mark Zuckerberg, new economy, Paul Graham, payday loans, Ponzi scheme, quantitative easing, Renaissance Technologies, Richard Feynman, risk tolerance, risk-adjusted returns, Robert Gordon, Robert Shiller, Robert Shiller, Ronald Reagan, Stephen Hawking, Steven Levy, stocks for the long run, the scientific method, traffic fines, Vanguard fund, working-age population

But the 7% of components that performed extremely well were more than enough to offset the duds. Just like Heinz Berggruen, but with Microsoft and Walmart instead of Picasso and Matisse. Not only do a few companies account for most of the market’s return, but within those companies are even more tail events. In 2018, Amazon drove 6% of the S&P 500’s returns. And Amazon’s growth is almost entirely due to Prime and Amazon Web Services, which itself are tail events in a company that has experimented with hundreds of products, from the Fire Phone to travel agencies. Apple was responsible for almost 7% of the index’s returns in 2018. And it is driven overwhelmingly by the iPhone, which in the world of tech products is as tail-y as tails get. And who’s working at these companies? Google’s hiring acceptance rate is 0.2%.²² Facebook’s is 0.1%.²³ Apple’s is about 2%.²⁴ So the people working on these tail projects that drive tail returns have tail careers.

Intuitively, you’d think the CEO should apologize to shareholders. But CEO Jeff Bezos said shortly after the disastrous launch of the company’s Fire Phone: If you think that’s a big failure, we’re working on much bigger failures right now. I am not kidding. Some of them are going to make the Fire Phone look like a tiny little blip. It’s OK for Amazon to lose a lot of money on the Fire Phone because it will be offset by something like Amazon Web Services that earns tens of billions of dollars. Tails to the rescue. Netflix CEO Reed Hastings once announced his company was canceling several big-budget productions. He responded: Our hit ratio is way too high right now. I’m always pushing the content team. We have to take more risk. You have to try more crazy things, because we should have a higher cancel rate overall. These are not delusions or failures of responsibility.


pages: 559 words: 155,372

Chaos Monkeys: Obscene Fortune and Random Failure in Silicon Valley by Antonio Garcia Martinez

Airbnb, airport security, always be closing, Amazon Web Services, Burning Man, Celtic Tiger, centralized clearinghouse, cognitive dissonance, collective bargaining, corporate governance, Credit Default Swap, crowdsourcing, death of newspapers, disruptive innovation, drone strike, El Camino Real, Elon Musk, Emanuel Derman, financial independence, global supply chain, Goldman Sachs: Vampire Squid, hive mind, income inequality, information asymmetry, interest rate swap, intermodal, Jeff Bezos, Kickstarter, Malcom McLean invented shipping containers, Marc Andreessen, Mark Zuckerberg, Maui Hawaii, means of production, Menlo Park, minimum viable product, MITM: man-in-the-middle, move fast and break things, move fast and break things, Network effects, orbital mechanics / astrodynamics, Paul Graham, performance metric, Peter Thiel, Ponzi scheme, pre–internet, Ralph Waldo Emerson, random walk, Ruby on Rails, Sam Altman, Sand Hill Road, Scientific racism, second-price auction, self-driving car, Silicon Valley, Silicon Valley startup, Skype, Snapchat, social graph, social web, Socratic dialogue, source of truth, Steve Jobs, telemarketer, undersea cable, urban renewal, Y Combinator, zero-sum game, éminence grise

Until AdGrok’s very end, search terms like “goldman sachs” and “fuck you” (I had written a post about the ever-elusive goal of “fuck-you money”) would be the most popular terms that led to clicks to our site.* It irritated MRM to no end. But hey—I didn’t see fifty thousand people a day lining up to use the product we had built. We’d take eyeballs wherever we could find them. As a result of the Amazon Web Services near fiasco, plus several more outages and server meltdowns, MRM suggested we run a “chaos monkey” from time to time. This was a software tool created and open-sourced by Netflix, meant to test a product or website’s resiliency against random server failures (such as we’d just witnessed with the blog). In order to understand both the function and the name of the chaos monkey, imagine the following: a chimpanzee rampaging through a data center, one of the air-conditioned warehouses of blinking machines that power everything from Google to Facebook.

Angels who used to write $20,000 checks on a deal could now easily write one for $200,000 or more (e.g., our boy Sacca). In tandem, the popularity of accelerators like Y Combinator, plus a general acceptance of entrepreneurship as a career, meant lots of very skilled engineers and product people were skipping the corporate trajectory and building exciting products. The emergence of turnkey, on-demand computation like Amazon Web Services, plus off-the-shelf Web-development frameworks like Ruby on Rails, meant that new ideas were easier than ever to test. Many entrepreneurs chose to build shovels rather than dig for gold, creating more complex software building blocks to underpin the innovation, such as back-end services like Parse, accelerating the startup explosion in an almost exponential way. The net of all this change was that seed rounds were now reaching levels of former A rounds; a two-month-old company with a persuasive CEO raising $2 million and calling it a “seed” was not shocking news.

Taking both technical and legal forms, it’s the snooping around an acquiring company does to make sure it’s actually getting what it thinks it is. On the technical side, it means understanding the company’s “stack”; that is, the pile of interrelated user interface and back-end server technologies that power the product. It might even be as detailed as line-by-line code reviews with the startup’s engineers. You can fake a lot in a startup these days, what with Amazon Web Services and all sorts of off-the-shelf back-end components that let any even minimally competent duffer set up a Web app that does something. Intelligent planning for growth is rare among early startups, but it’s the name of the game at a large, rapidly scaling tech company. Waiting for a team to grow from technical adolescence to mature talent was too long even for a larger company. As a first step, Twitter had invited us in as a group to talk technical turkey with a pack of engineers that reported to Kevin Weil.


pages: 215 words: 55,212

The Mesh: Why the Future of Business Is Sharing by Lisa Gansky

Airbnb, Amazon Mechanical Turk, Amazon Web Services, banking crisis, barriers to entry, carbon footprint, Chuck Templeton: OpenTable:, cloud computing, credit crunch, crowdsourcing, diversification, Firefox, fixed income, Google Earth, industrial cluster, Internet of things, Joi Ito, Kickstarter, late fees, Network effects, new economy, peer-to-peer lending, recommendation engine, RFID, Richard Florida, Richard Thaler, ride hailing / ride sharing, sharing economy, Silicon Valley, smart grid, social web, software as a service, TaskRabbit, the built environment, walkable city, yield management, young professional, Zipcar

These businesses are relatively easy to start and are spreading like wildfire: bike sharing, home exchanges, fashion swap parties, energy cooperatives, shared offices, cohousing, music studios, tool libraries, food and wine cooperatives, and many more. They leverage hundreds of billions of dollars in available information infrastructure—telecommunications, mobile technology, enhanced data collection, large and growing social networks, mobile SMS aggregators, and of course the Web itself. They efficiently employ horizontal business to business services, such as FedEx, UPS, Amazon Web Services, PayPal, and an ever-increasing number of cloud computing services. All the Mesh businesses rely on a basic premise: when information about goods is shared, the value of those goods increases, for the business, for individuals, and for the community. Mesh businesses are legally organized as for-profit corporations, cooperatives, and nonprofit organizations. Once I started looking, I quickly uncovered over 1,500 relevant companies and organizations.

Data-Rich, Highly Shareable Goods and Services Mesh Best. five flavors of the big Mesh. Not all large companies can adopt a Mesh strategy as thoroughly and successfully as Netflix. Many can, will, and have adopted aspects of the Mesh where they’re able to perceive the competitive advantages. Here are five ways: 1. Provide services or platforms that enable and encourage Mesh businesses. We’ve already discussed several, such as Amazon Web Services, PayPal, and FedEx. Like many other companies, the streaming music service Pandora extends its service through an iPhone app. The iPad, Kindle, and Kno are strong new platforms for distributing e-books, blogs, video, and magazines. These services are likely to be reshaped in even more favorable ways by the Mesh as it grows. A very important category of Mesh-enablers is the social networks, such as Twitter, Buzz, MySpace, LinkedIn, and Facebook. 2.


pages: 207 words: 57,959

Little Bets: How Breakthrough Ideas Emerge From Small Discoveries by Peter Sims

Amazon Web Services, Black Swan, Clayton Christensen, complexity theory, David Heinemeier Hansson, deliberate practice, discovery of penicillin, endowment effect, fear of failure, Frank Gehry, Guggenheim Bilbao, Jeff Bezos, knowledge economy, lateral thinking, Lean Startup, longitudinal study, loss aversion, meta analysis, meta-analysis, PageRank, Richard Florida, Richard Thaler, Ruby on Rails, Silicon Valley, statistical model, Steve Ballmer, Steve Jobs, Steve Wozniak, theory of mind, Toyota Production System, urban planning, Wall-E

Launched in 1999 and shut down in 2000, it suffered from customer service–related problems from the start. Critics ridiculed the company, calling it “Amazon.bomb” or “Amazon.con.” Some Wall Street analysts and investors even called for Bezos to resign. However, the ultimate outcome has been that Amazon’s exploratory mentality has spawned continual breakthroughs, such as Amazon stores, which allows small vendors to sell products on its site, as well as Amazon Web Services (AWS), which includes Elastic Compute Cloud (EC2), permitting third parties to rent storage space on the company’s servers. Third-party vendors now account for roughly 30% of Amazon’s sales, a key source of the company’s impressive growth. Chris Rock, the Google founders, and Jeff Bezos and his team are examples of people who approach problems in a nonlinear manner using little bets, what University of Chicago economist David Galenson has dubbed “experimental innovators.”

Abilities, 36–41 Academy Awards, 79, 145 Acoustics, 80–83 Active users, 131, 13–40 Adobe, 19, 89 Advertising, Internet, 4–5 AdWords, 4–5 Affordable loss principle, 28–32, 33 Afghanistan, 25–26, 105 Agile development, 84–91, 148, 151, 152, 194 Agilent Technologies, 19 Agriculture, 99 Alcoholism, 141–42 Al Jabouri, Najim Abdullah, 104, 105 Al-Qaeda, 25–26, 27, 92, 94 Amazon, 5–7, 45, 84, 107, 153 Amazon Auctions, 6 Amazon Web Services (AWS), 7 Ambition, 35 American Psychologist, 141 Animation, 29–32, 41–54, 59–62, 69–76, 105–106, 142–46, 148–49, 155, 158–59, 186–87 humor and, 73–76 plussing, 69–73 prototyping, 59–62 small wins, 142–46, 148–49, 150 See also Pixar; specific animated films Ansari, Daniel, 67–68 Apple, 12, 29, 30, 108, 156 Architecture, 13, 45–46, 54, 55, 77–83, 84, 117, 154, 161, 162 imposition of constraints, 77–84 prototyping, 54–55, 63, 80, 81 See also specific architects and buildings Ariel Ultra, 113–14 Army, U.S., 22–28, 91–95, 103–105, 149–51, 155 counterinsurgency methods, 27, 91–95, 103–105, 149–150, 151, 155 Ballmer, Steve, 156 Bangladesh, 98–102 Barnholt, Ned, 19–22 Bayles (David) and Orland (Ted), Art & Fear: Observations on the Perils (and Rewards) of Artmaking, 169 Beck, Kent, 84 Becket, Welton, 79 Beethoven, Ludwig van, 8, 13, 181–82 Begum, Sufiya, 98–102 Belkin Corporation, 111–13, 153 Belsky, Scott, Making Ideas Happen, 172–73 Berkowitz, Aaron, 67–68 Bezos, Jeff, 5–6, 7, 8, 13, 35, 45, 107, 110–11, 115, 153, 159 Biaj, Iraq, 93–94 Big bets, 20 little bets vs., 19–33 Bird, Brad, 42–44 Blank, Steve, 102–103 Bleeding edge user needs, 136 Blocking, 70 Body language, 123 Boston, 109 Brafman, Ori, 138, 139–40 Click, 138 Sway, 138 Brain, 15, 47, 66, 183–84 capabilities, 47–48 improvisation and, 66–68 Brereton, Kevin, 115 Brin, Sergey, 4–5, 8, 115 Brown, Tim, Change by Design, 171 Bug’s Life, A (film), 59 Bush, George W., 125, 127 Business Land, 112 Calculators, 21–22 Canon, 85 Cars (film), 106 Catmull, Ed, 30–32, 41–45, 60, 62, 75, 121, 142–46, 148–49, 150 CAT scans, 30 Central planning approach, 15–17, 24–26 Challenges, 38, 48, 49, 78 growth mind-set and, 35–49 Chance opportunities, 123, 124, 129 Chandler Pavilion, Los Angeles, 79, 80, 83 Change, 160–61 Chanos, James, 109 Chesborough, Henry, Open Innovation, 166 Chicago, 55–58 Child, Julia, 115 Child development, 160–61 Christensen, Clayton, 107 The Innovator’s Dilemma, 166 The Innovator’s Solution, 166 Clinton, Hillary, 56, 125, 126 CNBC, 125 Cockburn, Alistair, 84 Cold War, 23–26 Collins, Jim: Built to Last, 166–67 Good to Great, 120–21, 166 How the Mighty Fall, 166 Columbia University, 36 Comedian, The (film), 165 Comedians, 1–3, 12, 13, 32, 109, 128, 131, 133, 135, 140, 141, 151, 154–55 Competition, 17, 156 Computer Land, 112 Computer Point, 112 Computers, 22, 29, 83, 107–108, 110, 112–13, 142–46 animation, 29–32, 41–45, 59–62, 69–76, 105–106, 142–46, 148–49, 186–87 software, 83–91, 112, 136–37, 143–46, 149, 151–52 See also Internet Conceptual innovation, 7, 180–81 Confidence, and growth mind-set, 35–41 Consolidating gains, 149 Constraints, imposition of.


pages: 382 words: 105,819

Zucked: Waking Up to the Facebook Catastrophe by Roger McNamee

4chan, Albert Einstein, algorithmic trading, AltaVista, Amazon Web Services, barriers to entry, Bernie Sanders, Boycotts of Israel, Cass Sunstein, cloud computing, computer age, cross-subsidies, data is the new oil, Donald Trump, Douglas Engelbart, Douglas Engelbart, Electric Kool-Aid Acid Test, Elon Musk, Filter Bubble, game design, income inequality, Internet of things, Jaron Lanier, Jeff Bezos, John Markoff, laissez-faire capitalism, Lean Startup, light touch regulation, Lyft, Marc Andreessen, Mark Zuckerberg, market bubble, Menlo Park, Metcalfe’s law, minimum viable product, Mother of all demos, move fast and break things, move fast and break things, Network effects, paypal mafia, Peter Thiel, pets.com, post-work, profit maximization, profit motive, race to the bottom, recommendation engine, Robert Mercer, Ronald Reagan, Sand Hill Road, self-driving car, Silicon Valley, Silicon Valley startup, Skype, Snapchat, social graph, software is eating the world, Stephen Hawking, Steve Jobs, Steven Levy, Stewart Brand, The Chicago School, Tim Cook: Apple, two-sided market, Uber and Lyft, Uber for X, uber lyft, Upton Sinclair, WikiLeaks, Yom Kippur War

With open source stacks as a foundation, engineers could focus all their effort on the valuable functionality of their app, rather than building infrastructure from the ground up. This saved time and money. In parallel, a new concept emerged—the cloud—and the industry embraced the notion of centralization of shared resources. The cloud is like Uber for data—customers don’t need to own their own data center or storage if a service provides it seamlessly from the cloud. Today’s leader in cloud services, Amazon Web Services (AWS), leveraged Amazon.com’s retail business to create a massive cloud infrastructure that it offered on a turnkey basis to startups and corporate customers. By enabling companies to outsource their hardware and network infrastructure, paying a monthly fee instead of the purchase price of an entire system, services like AWS lowered the cost of creating new businesses and shortened the time to market.

The case against Amazon is probably strongest, and it provides a framework for understanding the larger issues. For Amazon.com, freedom from antitrust scrutiny has allowed the company to integrate vertically, as well as horizontally. From its original base in retail for nonperishable goods, Amazon has expanded horizontally into perishables, with Whole Foods, and into cloud services, with Amazon Web Services. Amazon’s vertical integration has included Marketplace, which incorporates third-party sellers; Basics, where Amazon private-labels bestselling commodity products; and hardware, such as Alexa voice-controlled devices and the Fire home video server. In a traditional antitrust regime, Amazon’s vertical-integration strategy would not be allowed. The use of proprietary consumer data to identify, develop, and sell products in direct competition with bestsellers on the site represents an abuse of power that would have appalled regulators prior to 1981.

Accel Partners, 54, 58 Acxiom, 226 addiction, 100–101, 106–7, 162, 206, 240, 246, 250, 253, 254, 257, 268, 269, 281 advertising, 17, 47, 68–69, 85–86, 105, 120, 125, 257, 283–84 Facebook and, 11–12, 47, 59–61, 63, 68–77, 85, 103, 119, 128–29, 130, 132, 143, 148, 173, 184, 185, 202, 207–9, 211, 217–19, 237–38, 258, 265, 270, 281 Google and, 47, 67–69, 86, 104, 173, 283–86 political, microtargeting in, 237–38 YouTube and, 103, 283–84 Albright, Jonathan, 125 Alcorn, Al, 34 algorithms, 66, 94, 98, 122, 128, 129, 169, 264 extreme views and, 92 of Facebook, 4, 9, 11, 66, 74, 76, 81, 87, 91, 128–29, 143, 166, 232, 235, 243, 270, 274, 277, 281 of Google, 66, 235 of YouTube, 92–93, 139, 274 Allen, Paul, 25 Altair, 34 Altavista, 42 Alter, Adam, 272 Amanpour, Christiane, 147 Amazon, 16, 27, 36, 38, 68, 137, 224, 262, 283 Alexa, 137, 262, 268, 271, 281 Echo, 236 market power of, 46 monopoly power and antitrust issues, 47–48, 136–40, 225–26, 246, 247, 261, 262, 284 Web Services, 40, 41, 137, 138 words associated with, 231 American Academy of Pediatrics, 273 Anderson, Chris, 109 Anderson, Fred, 14 Andreessen, Marc, 27, 36, 37, 43, 58 Android, 138, 204, 271, 282 angel investors, 48 anonymity, 37, 55, 92, 101, 123 antitrust and monopoly issues, 46–48, 136–41, 162, 220, 223, 224, 227, 234, 238, 246, 247, 250, 261–63, 277, 279, 281–83, 285–87 Amazon and, 47–48, 136–40, 225–26, 246, 247, 261, 262, 284 AT&T and, 224–25 Facebook and, 47–48, 100, 136–41, 162, 225–26, 234, 246, 247, 261, 263, 284 Google and, 47–48, 136–40, 162, 225–26, 234, 246, 247, 261–63, 282, 284 Microsoft and, 46–47, 154–55, 286 Apache, 40 Apple, 14, 25, 29–30, 34, 35, 38, 49, 50, 83, 118, 249 iPhone, 38, 84, 100, 105–6, 271 Jones and Infowars and, 228–29 market power of, 46 Onavo and, 140 privacy and, 38, 158, 271 Silver Lake and, 29–30 words associated with, 231 approval, need for, 98 Apture, 107 Arab Spring, 64, 243 ARPANET, 34–35 artificial intelligence (AI), 69, 84–85, 88, 98, 128, 151, 158, 219–20, 223, 236, 252, 260, 261, 263, 264, 267–69, 272 of Facebook, 10, 11, 69, 85, 87, 91, 95, 108, 203, 219–20, 230, 261 of Google, 108, 219–20, 253, 261 Associated Press, 124 AT&T, 224–25 Atari, 34 Atlantic, 195, 231 Attention Merchants, The (Wu), 120 attorneys general, 120, 172, 227 authoritarianism, 278, 279 automobiles, 158, 223 banks, 231–32 Bannon, Steve, 181, 182, 197, 199 Bast, Andy, 109 Belgium, 247 Bell Labs, 225 Berners-Lee, Tim, 36, 161 Bezos, Jeff, 27 Black Lives Matter, 8, 243, 250, 275 Blacktivist, 131 Bloomberg, 50 Blumenthal, Richard, 128 Bodnick, Marc, 14, 18, 59 Bogost, Ian, 195–96 Bolton, Tamiko, 161–62 Bono, 13, 18, 28–29, 30, 59–61, 159 Booker, Cory, 128 Bosworth, Andrew “Boz,” 160, 165 memo of, 204–6 bots, 90, 116, 124, 126, 174, 177, 227 bottomless bowl, 97 Box, 41 brain, 88 brain hacking, 81–82, 118, 141, 148 Brand, Stewart, 177 Brazil, 280 Brexit, 8–9, 96, 180, 196, 198, 244 Breyer, Jim, 58 Bricklin, Dan, 34 Brin, Sergey, 27 bro and hipster cultures, 49–50 Brotopia (Chang), 50 bullying, 87, 101–2, 205, 253, 269, 273, 280, 281 Bushnell, Nolan, 34 Business Insider, 141 BuzzFeed, 204 California, 227–28 environmental regulations in, 201 secession movement, 114, 115 Cambodia, 215, 246 Cambridge Analytica, 78, 180–98, 199, 202–4, 207, 208, 210, 213, 216–18, 251, 259 Cambridge University, 181 Candy Crush, 191, 269 capitalism, 200, 201, 220, 238, 262 Castor, Kathy, 210–11 cell phones, 36, 225 see also smartphones censorship, 252 Center for Humane Technology (CHT), 157, 166–67, 173, 188, 272 Chancellor, Joseph, 181, 186 Chang, Emily, 50 Chaos Monkeys (García Martínez), 71, 72, 73 Chaslot, Guillaume, 92 Chicago School antitrust philosophy, 137–39, 285, 286 children, 214, 240, 253–54 technology and, 106, 156, 166, 237, 255, 268, 269, 272–73, 279–80 China, 162, 205, 215–16 Cisco, 46 CityVille, 191 Clayton Act, 136 Clinton, Bill, 60 Clinton, Chelsea, 167 Clinton, Hillary, 5, 11, 117, 121, 124, 125, 130, 166 Facebook Groups and, 7–8 Clinton Foundation, 130 cloud, 38, 40, 41, 104, 249, 250 Amazon Web Services, 40, 41, 137, 138 CNBC, 118, 161 CNN, 130–31, 147, 161, 193, 229, 231 Coca-Cola, 141 Cohen, Michael, 209–10 Cold War, 32 Columbia University, 54 Common Sense Media, 119, 156–57, 167, 227, 272 Compaq Computer, 27, 35, 152 competition: lack of alteratives to Facebook and Google, 92, 100, 141, 223, 280 see also antitrust and monopoly issues computers, computing, 22, 25, 31–36, 108, 225 in classrooms, 273 cloud, see cloud minicomputers, 33, 46 PCs, 17, 22, 25, 29, 33–36, 40, 42, 45, 46, 273 Congress, 121–33, 135–36, 152, 154, 163, 200, 208, 222, 226, 238 House Energy and Commerce Committee, 209–10, 227 House Intelligence Committee, 123, 127–28, 132, 133, 167, 227 Senate Commerce Committee, 209 Senate Intelligence Committee, 12, 111–12, 127, 132 Senate Judiciary Committee, 128, 131, 132, 136, 209 Zuckerberg’s testimony before, 209–12, 216, 217 Conservative Political Action Conference (CPAC), 174 conspiracy theories, 92–95, 115, 119, 121, 214, 228, 229, 234, 242, 243, 274 Pizzagate, 124–26, 130 Constitution, 12, 259 Fourth Amendment to, 201 content vendors, 283 Conway, Ron, 48 Cooper, Anderson, 81 copyrights, 281 Cosgrave, Paddy, 108–9 Cow Clicker, 195–96 Cox, Chris, 144 Cox, Joseph, 229–30 Cruz, Ted, 185 Currier, James, 47 Cyprus, 125 Czech Republic, 125 Dalai Lama, 31 dark patterns, 96 data and data privacy, 155, 158, 159, 203, 217, 220, 234–36, 238, 253, 258–59, 263–65, 269, 271–72, 277–84 Apple and, 38, 158, 271 Big Data, 158 browsing history, 218–19 Cambridge Analytica and, 78, 180–98, 199, 202–4, 207, 208, 210, 213, 216–18, 251, 259 combined sets of, 68, 285–86 and data as currency, 285 data ownership, 222, 237, 247, 259, 264 data portability, 247 Facebook’s banning of data brokers, 208 Facebook’s continuing threat to, 246 Facebook user data, 4–5, 9, 62, 72, 75–76, 78, 87, 131–32, 141–42, 174, 180–98, 202–4, 210–11, 216–19, 223, 258–59 Facebook user privacy settings, 97 fiduciary rule and, 226–27, 247, 260–61 Global Data Protection Regulation, 221, 222, 224, 259–60 and Internet of Things, 262, 268, 271 and internet as barter transaction, 285 internet privacy bill of rights, 221–24, 226–27 metadata, 68–69, 211, 217–20, 264 passwords and log-ins, 249–50, 271 “price” of data, 285–86 regulations and, 201 and value of data vs. services, 286 Zuckerberg’s attitudes regarding, 4–5, 55–56, 60, 141–42 Data for Democracy, 90, 122, 127 Dead & Company, 74 Defense, U.S.


Remix: Making Art and Commerce Thrive in the Hybrid Economy by Lawrence Lessig

Amazon Web Services, Andrew Keen, Benjamin Mako Hill, Berlin Wall, Bernie Sanders, Brewster Kahle, Cass Sunstein, collaborative editing, commoditize, disintermediation, don't be evil, Erik Brynjolfsson, Internet Archive, invisible hand, Jeff Bezos, jimmy wales, Joi Ito, Kevin Kelly, Larry Wall, late fees, Mark Shuttleworth, Netflix Prize, Network effects, new economy, optical character recognition, PageRank, peer-to-peer, recommendation engine, revision control, Richard Stallman, Ronald Coase, Saturday Night Live, SETI@home, sharing economy, Silicon Valley, Skype, slashdot, Steve Jobs, The Nature of the Firm, thinkpad, transaction costs, VA Linux, yellow journalism

In January 2007 the company began Amapedia, “a collaborative wiki for user-generated content related to ‘the products you like the most.’ ”10 In the decade since Amazon launched, it has delivered to the market an extraordinary range of innovation. Everything it does is aimed to drive sales of its products more efficiently. One of the techniques that Amazon uses mirrors the technique of the Internet generally: Amazon has opened its platform to allow others to innovate in new ways to build value out of Amazon’s database. Through a suite of tools called Amazon Web Services (AWS), Amazon enables developers to build products that integrate directly into Amazon’s database. For example, a developer named Jim Biancolo used AWS to build a free Web tool to track the price difference between new products and used products (plus shipping). And a company called TouchGraph used AWS to build a product browser that would show the links between related products. Enter Cass Sunstein’s, for example, and you’ll see all the books in Amazon that relate to Sunstein’s books in subject and citation.

Its purpose is to “improve the accuracy of predictions about how much someone is going to love a movie based on their movie preferences.”27 To achieve this end, Netflix runs a “Netflix Prize”—offering a grand prize of $1 million to anyone who improves Netflix’s own system by more than 10 percent. To enable this competition to happen, Netflix shared “a lot of anonymous rating data.” The company 80706 i-xxiv 001-328 r4nk.indd 137 8/12/08 1:55:20 AM REMI X 138 also increasingly offers through RSS feeds access to ranking information about its users’ choices. Amazon does this through its Amazon Web Services. And Google does this perhaps most of all, through Google APIs that encourage what has come to be known as the Google mash-up. Don Tapscott and Anthony Williams describe one example of the Google mash-up in their book, Wikinomics. In May 2005, Paul Rademacher was trying to find a house in Silicon Valley for his job at Dreamworks Animation. He grew weary of the piles of Google maps for each and every house he wanted to see, so he created a new Web site that cleverly combines listings from the online classified-ad service craigslist with Google’s mapping service.

., 195. 80706 i-xxiv 001-328 r4nk.indd 318 8/12/08 1:56:29 AM INDEX ABC, 213 Academy Awards, 45–46 access, 43–49, 67, 106, 255, 261, 291 advertising, 2, 48–49 blogs and, 63, 291 classified, 187 Dogster and, 187 Wikia and, 204–5 Wikipedia and, 161–62, 203–4 Alcoholics Anonymous (AA), 148 Alexa, 236–37 Alexander, Alastair, 208 Allman, Eric, 163–64 amateur creativity, 33, 90, 103 deregulation of, 254–59 in music, 24–29, 32–33 Amazon, 48–49, 125–26, 129–30, 141, 142, 239–40 Little Brother and, 132, 136 Amazon Web Services (AWS), 126, 138 Anderson, Chris, 129 Andover.net, 199 anime music videos (AMVs), 77–78, 79, 80 AOL, 153–54 Apache, 164–65, 183, 241–42 Apple, 88, 142, 165 iPod, 41, 46, 47, 88 iTunes, 12, 13, 41–42, 134 Armstrong, Edwin Howard, 30 Armstrong, Louis, 104 80706 i-xxiv 001-328 r4nk.indd 319 astronomy, 170–71 Atmo, 73 AudioMulch, 12 Awesometown, 227 Bainwol, Mitch, 114 Baker, Stewart, 89 Barish, Stephanie, 80 Barlow, John Perry, 67 barter economies, 180 Baumol, William, 230 Beam-it, 135 Beatles, 74, 75, 255 Becker, Don, 180 Behlendorf, Brian, 164, 183, 242–43 Benkler, Yochai, 50, 58, 59, 62, 118, 140, 146, 169, 172, 176, 178 Berners-Lee, Tim, 58, 221 Bezos, Jeff, 125, 129, 136 Biancolo, Jim, 126 BigChampagne Online Media Measurement, 110 BIND, 164 Blair, Tony, 73–74, 273 blip.tv, 249 Blockbuster, 123–24, 129, 142 blogs, 57, 58–62, 63–65, 103, 139, 291 value of, 92–93 books, 42, 99–100, 268, 269, 291, 292 access to, 43–44 8/12/08 1:56:29 AM 320 books (cont.)


pages: 540 words: 103,101

Building Microservices by Sam Newman

airport security, Amazon Web Services, anti-pattern, business process, call centre, continuous integration, create, read, update, delete, defense in depth, don't repeat yourself, Edward Snowden, fault tolerance, index card, information retrieval, Infrastructure as a Service, inventory management, job automation, Kubernetes, load shedding, loose coupling, microservices, MITM: man-in-the-middle, platform as a service, premature optimization, pull request, recommendation engine, social graph, software as a service, source of truth, the built environment, web application, WebSocket

You can target scaling at just those microservices that need it Gilt, an online fashion retailer, adopted microservices for this exact reason. Starting in 2007 with a monolithic Rails application, by 2009 Gilt’s system was unable to cope with the load being placed on it. By splitting out core parts of its system, Gilt was better able to deal with its traffic spikes, and today has over 450 microservices, each one running on multiple separate machines. When embracing on-demand provisioning systems like those provided by Amazon Web Services, we can even apply this scaling on demand for those pieces that need it. This allows us to control our costs more effectively. It’s not often that an architectural approach can be so closely correlated to an almost immediate cost savings. Ease of Deployment A one-line change to a million-line-long monolithic application requires the whole application to be deployed in order to release the change.

The problem, of course, is that if the same people create both the server API and the client API, there is the danger that logic that should exist on the server starts leaking into the client. I should know: I’ve done this myself. The more logic that creeps into the client library, the more cohesion starts to break down, and you find yourself having to change multiple clients to roll out fixes to your server. You also limit technology choices, especially if you mandate that the client library has to be used. A model for client libraries I like is the one for Amazon Web Services (AWS). The underlying SOAP or REST web service calls can be made directly, but everyone ends up using just one of the various software development kits (SDKs) that exist, which provide abstractions over the underlying API. These SDKs, though, are written by the community or AWS people other than those who work on the API itself. This degree of separation seems to work, and avoids some of the pitfalls of client libraries.

It wanted teams to own and operate the systems they looked after, managing the entire lifecycle. But Amazon also knew that small teams can work faster than large teams. This led famously to its two-pizza teams, where no team should be so big that it could not be fed with two pizzas. This driver for small teams owning the whole lifecycle of their services is a major reason why Amazon developed Amazon Web Services. It needed to create the tooling to allow its teams to be self-sufficient. Netflix learned from this example, and ensured that from the beginning it structured itself around small, independent teams, so that the services they created would also be independent from each other. This ensured that the architecture of the system was optimized for speed of change. Effectively, Netflix designed the organizational structure for the system architecture it wanted.


pages: 628 words: 107,927

Node.js in Action by Mike Cantelon, Marc Harter, Tj Holowaychuk, Nathan Rajlich

Amazon Web Services, Chris Wanstrath, create, read, update, delete, Debian, en.wikipedia.org, Firefox, Google Chrome, MITM: man-in-the-middle, MVC pattern, node package manager, p-value, pull request, Ruby on Rails, web application, WebSocket

Figure 12.1 illustrates how you can use cloud hosting to automate the creation and destruction of an application’s servers. Figure 12.1. Creating, starting, stopping, and destroying cloud servers can be fully automated. The downside to using cloud servers is that they tend to be more expensive than VPSs and can require some knowledge specific to the cloud platform. Amazon Web Services The oldest and most popular cloud platform is Amazon Web Services (AWS; http://aws.amazon.com/). AWS consists of a range of different hosting-related services, like email delivery, content-delivery networks, and lots more. Amazon’s Elastic Compute Cloud (EC2), one of AWS’s central services, allows you to create servers in the cloud whenever you need them. EC2 virtual servers are called instances, and they can be managed using either the command line or a web-based control console, shown in figure 12.2.

Index [SYMBOL][A][B][C][D][E][F][G][H][I][J][K][L][M][N][O][P][Q][R][S][T][U][V][W][X] SYMBOL 404 status code, 2nd 500 status code, 2nd A absolute paths vs. relative paths Accept header acceptance testing defined overview Soda installing overview using with Sauce Labs using with Selenium Server Tobi Accept-Encoding header add command add-ons for flow control adduser command administration panel middleware after function, 2nd afterEach function, 2nd Ajax (Asynchronous JavaScript and XML) Amazon S3 Amazon Web Services ANSI (American National Standards Institute) ANSI escape codes Apache application dependency application/x-www-form-urlencoded content type, 2nd, 3rd arguments, command-line argv property, 2nd ascii encoding ASI (automated semicolon insertion) asynchronous I/O Asynchronous JavaScript and XML. See Ajax. attributes, for tags in Jade authenticating users authentication middleware logging in users authenticating logins creating menu for authenticated users displaying login form registering users creating registration form creating routes implementing relaying feedback to users storing messages in sessions storing user data authenticating user logins creating json file creating model for retrieving user data saving user into Redis securing passwords testing user-saving logic user loading middleware authenticity token Authorization header, 2nd automated semicolon insertion.


pages: 31 words: 9,168

Designing Reactive Systems: The Role of Actors in Distributed Architecture by Hugh McKee

Amazon Web Services, fault tolerance, Internet of things, microservices

When programs run in a computer system, the actual work is handled by what is called a thread or thread of execution. Just as each computer system has a limited number of cores and a limited amount of memory, there is also a limited number of threads. A typical computer system has a small number of cores, and common small or medium-sized servers may have 4 to 16 processing cores. (Just looking at Amazon Web Services instances, you can see it’s possible to provision cloud instances with 128 processor cores and 2 terabytes of in-memory RAM.) For memory (RAM), this varies from a few gigabytes to terabytes. A typical server has 8GB to 64GB, but this number keeps going up as the cost of memory decreases in light of technological improvements. For threads, the count typically falls into the hundreds range.


Seeking SRE: Conversations About Running Production Systems at Scale by David N. Blank-Edelman

Affordable Care Act / Obamacare, algorithmic trading, Amazon Web Services, bounce rate, business continuity plan, business process, cloud computing, cognitive bias, cognitive dissonance, commoditize, continuous integration, crowdsourcing, dark matter, database schema, Debian, defense in depth, DevOps, domain-specific language, en.wikipedia.org, fault tolerance, fear of failure, friendly fire, game design, Grace Hopper, information retrieval, Infrastructure as a Service, Internet of things, invisible hand, iterative process, Kubernetes, loose coupling, Lyft, Marc Andreessen, microservices, minimum viable product, MVC pattern, performance metric, platform as a service, pull request, RAND corporation, remote working, Richard Feynman, risk tolerance, Ruby on Rails, search engine result page, self-driving car, sentiment analysis, Silicon Valley, single page application, Snapchat, software as a service, software is eating the world, source of truth, the scientific method, Toyota Production System, web application, WebSocket, zero day

This is especially important because RUM can expose personally identifiable information (PII), cookies, and IP details. This might run afoul of the EU’s General Data Protection Regulation (GDPR)8 compliance rules, which went into effect on May 25, 2018. Again, here are some pitfalls to avoid: Don’t reinvent the wheel; if you have a Data Science or other team concerned about RUM, it might already have a data pipeline9 that we can utilize. Similarly, Google, Amazon Web Services (AWS), and others offer solutions for message queueing and streaming. Don’t rely on third-party APIs as a conduit for monitoring third-party health. Third parties generally do an excellent job of performing what they’re paid to do; however, their reporting APIs are not necessarily their core competency. If you rely on third-party reporting APIs, you are likely relying on data that is stale or inaccurate.

To maximize the effectiveness of self-service, the ability to both define and execute automated procedures needs to be provided, as demonstrated in Figure 10-7. Of course, you need to put constraints in place to enforce security boundaries and help prevent mistakes, but providing the ability to define and execute delivers the most value. Figure 10-7. Traditional ticket-driven request fulfillment versus full self-service For example, consider the Elastic Compute Cloud (EC2) service from Amazon Web Services (AWS). The ability to press a button and get a running virtual machine was interesting. However, the ability to control your own destiny by making your own machine images (AMIs) and configuration was revolutionary. It empowered individuals and allowed teams to decouple and move at their own pace. However, it isn’t unfettered access. Users are constrained for security reasons by both AWS and their own self-selected security policies.

Feel free to get in touch if you’d like to share your ideas, too, I’d love to hear them. Replies Site reliability is operational reliability, scalability, and efficiency. This includes business continuity (disaster recovery, high availability). The operational site becomes a product in and of itself and may include their own CI/CD for internal tooling. The automation tends toward custom tools; for example, Python with Boto library, Ruby with AWS [Amazon Web Services] SDK, and Go language rather than the use of high-level tools like Terraform and Ansible, as they are considered inefficient. Though this is not absolute, just a trend. SRE is programming the operations to create reliable and efficient infrastructure. DevOps focuses on breaking down cultural silos and increasing efficiency or velocity of deployment (CI/CD) pipeline, from development to delivery; this includes building and pre/post artifact testing (testing before and after the artifact is built), thus CT, or continual testing.


pages: 274 words: 58,675

Puppet 3 Cookbook by John Arundel

Amazon Web Services, cloud computing, continuous integration, Debian, defense in depth, DevOps, don't repeat yourself, GnuPG, Larry Wall, place-making, Ruby on Rails, web application

If you had to build the servers by hand every time, this process would be too lengthy to make it worthwhile, but with automated configuration management, it's a snap. There are plenty of cloud service providers around, with Amazon's EC2 being one of the best-known and oldest-established. In this recipe I'll show you how to manage cloud instances using EC2, but the general principles and most of the code will be adaptable to any cloud provider. Getting ready… You'll need an Amazon Web Services (AWS) account if you don't already have one. Of course if you're using an EC2 instance to try out the recipes in this book, you'll already have everything you need, but otherwise you can sign up for an AWS account here: http://aws.amazon.com/ You'll need the AWS access key ID and secret access key corresponding to your account. You can find these on the Security Credentials page of the AWS portal.

How it works… puppet config print will output every configuration parameter and its current value (and there are lots of them). To see the value for a specific parameter, add it as an argument to the puppet config print command: ubuntu@cookbook:~/puppet$ puppet config print noop false See also The Generating reports recipe in this chapter 251 Index A B a2ensite command 148 Amazon EC2 tab 188 Amazon Web Services (AWS) 188 Apache servers about 144 managing 144, 145 Apache virtual hosts creating 145-147 custom domains 149 docroots 149 working 148, 149 APT package framework 106 arguments passing, to shell commands 84, 85 array iteration using, in templates 96, 97 arrays about 45 creating 45 creating, with split function 46 hashes, using 46 using 45 working 45 arrays, of resources using 58 audit metaparameter 140 Augeas tool about 87 automatically editing config files with 89, 90 auto_failback setting 171 automatic syntax checking Git hooks, used 29 AWS Management Console URL 188 Blueprint installing 214 running 215, 216 C case statements about 50, 52 default value, specifying 52 examples 50 working 51 CentOS 8 changes parameter 90 check_syntax.sh script 32 circular dependency about 245 fixing 248 class inheritance extra values, adding using +> operator 78 using 75, 76 cloud computing 188 command output logging 237, 238 working 238 community Puppet style about 34 indentation 34 parameters 35 quoting 34 symlinks 36 using 34 variables 35 conditional statements comparisons 48 elsif branches 48 example 47 expressions, combining 48 working 47 writing 47 config files building with snippets 91- 93 editing automatically with Augeas 89-91 adding lines to 88, 89 configuration data importing, with Hiera 206-208 configuration settings inspecting 251 contain_package assertion 220 cron Puppet, running from 18, 19 cron jobs distributing 126-128 cross-platform manifests writing 79 custom facts creating 200, 201 D Dean Wilson example, URL 231 debug messages logging 239 resource, checking 240 variable values, printing 240 decentralized Puppet architecture creating 14, 16 definitions creating 59 using 59 working 60 dependencies using 61- 64 dependency graphs drawing 245-247 directory trees distributing 135, 136 DocumentRoot parameter 150 dotfiles 121 DOT format, graphs expanded_relationships.dot 247 254 relationships.dot 247 resources.dot 247 dry run mode using 236, 237 dynamic information importing 83 E EC2 instances AWS example, creating 188-193 managing 188 ENC.


pages: 561 words: 157,589

WTF?: What's the Future and Why It's Up to Us by Tim O'Reilly

4chan, Affordable Care Act / Obamacare, Airbnb, Alvin Roth, Amazon Mechanical Turk, Amazon Web Services, artificial general intelligence, augmented reality, autonomous vehicles, barriers to entry, basic income, Bernie Madoff, Bernie Sanders, Bill Joy: nanobots, bitcoin, blockchain, Bretton Woods, Brewster Kahle, British Empire, business process, call centre, Capital in the Twenty-First Century by Thomas Piketty, Captain Sullenberger Hudson, Chuck Templeton: OpenTable:, Clayton Christensen, clean water, cloud computing, cognitive dissonance, collateralized debt obligation, commoditize, computer vision, corporate governance, corporate raider, creative destruction, crowdsourcing, Danny Hillis, data acquisition, deskilling, DevOps, Donald Davies, Donald Trump, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Filter Bubble, Firefox, Flash crash, full employment, future of work, George Akerlof, gig economy, glass ceiling, Google Glasses, Gordon Gekko, gravity well, greed is good, Guido van Rossum, High speed trading, hiring and firing, Home mortgage interest deduction, Hyperloop, income inequality, index fund, informal economy, information asymmetry, Internet Archive, Internet of things, invention of movable type, invisible hand, iterative process, Jaron Lanier, Jeff Bezos, jitney, job automation, job satisfaction, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, Kevin Kelly, Khan Academy, Kickstarter, knowledge worker, Kodak vs Instagram, Lao Tzu, Larry Wall, Lean Startup, Leonard Kleinrock, Lyft, Marc Andreessen, Mark Zuckerberg, market fundamentalism, Marshall McLuhan, McMansion, microbiome, microservices, minimum viable product, mortgage tax deduction, move fast and break things, move fast and break things, Network effects, new economy, Nicholas Carr, obamacare, Oculus Rift, packet switching, PageRank, pattern recognition, Paul Buchheit, peer-to-peer, peer-to-peer model, Ponzi scheme, race to the bottom, Ralph Nader, randomized controlled trial, RFC: Request For Comment, Richard Feynman, Richard Stallman, ride hailing / ride sharing, Robert Gordon, Robert Metcalfe, Ronald Coase, Sam Altman, school choice, Second Machine Age, secular stagnation, self-driving car, SETI@home, shareholder value, Silicon Valley, Silicon Valley startup, skunkworks, Skype, smart contracts, Snapchat, Social Responsibility of Business Is to Increase Its Profits, social web, software as a service, software patent, spectrum auction, speech recognition, Stephen Hawking, Steve Ballmer, Steve Jobs, Steven Levy, Stewart Brand, strong AI, TaskRabbit, telepresence, the built environment, The Future of Employment, the map is not the territory, The Nature of the Firm, The Rise and Fall of American Growth, The Wealth of Nations by Adam Smith, Thomas Davenport, transaction costs, transcontinental railway, transportation-network company, Travis Kalanick, trickle-down economics, Uber and Lyft, Uber for X, uber lyft, ubercab, universal basic income, US Airways Flight 1549, VA Linux, Watson beat the top human players on Jeopardy!, We are the 99%, web application, Whole Earth Catalog, winner-take-all economy, women in the workforce, Y Combinator, yellow journalism, zero-sum game, Zipcar

Yet for Instagram to exist and thrive, every phone had to include a digital camera and to be connected to a communications network, and that network had to be pervasive and data centers had to provide hosting services that allow tiny startups to serve tens of millions of users. (Instagram had perhaps 40 million users when it was bought; it has 500 million today.) Add up the employees of Apple and Samsung, Cisco and Huawei, Verizon and AT&T, Amazon Web Services (where Instagram was originally hosted) and Facebook’s own data centers, and you see the size of the mountain range of employment of which Instagram itself is a boulder on one small peak. But that’s not all. These digital communications and content creation technologies have made it possible for a new class of media company—Facebook, Instagram, YouTube, Twitter, Snap, WeChat, Tencent, and a host of others around the world—to turn ordinary people into “workers” producing content for their advertising business.

He jumped up in the back of the room when I was done and said, “You didn’t say the bit about a platform beating an application every time!” I didn’t make that mistake when I gave another version of the talk at an Amazon all-hands meeting in May 2003. The first-generation web services that the e-commerce giant rolled out in 2003 were all about access to their in-house product catalog and its underlying data, and had little to do with the infrastructure services that were launched in 2006 under the name Amazon Web Services (or AWS) and that sparked the great industry transformation that is now called “cloud computing.” Those services came about for entirely different reasons, but I like to think that I planted the seeds of the idea with Jeff that if Amazon was to prosper in the years ahead, it had to become far more than just an e-commerce application. It had to become a platform. In that marvelous way he has of taking any idea and thinking it all the way through, Jeff took the idea of platform much further than I had imagined.

As Jeff described in a short 2008 interview with Om Malik: “Four years ago is when it started, and we had enough complexity inside Amazon that we were finding we were spending too much time on fine-grained coordination between our network engineering groups and our applications programming groups. Basically what we decided to do is build a [set of APIs] between those two layers so that you could just do coarse-grained coordination between those two groups.” (That is, “small pieces loosely joined.”) This is important: Amazon Web Services was the answer to a problem in organizational design. Jeff understood, as every network-enabled business needs to understand in the twenty-first century, that, as HR consultant Josh Bersin once said to me, “Doing digital isn’t the same as being digital.” In the digital era, an online service and the organization that produces and manages it must become inseparable. How Jeff took the idea of Amazon as a platform out of the realm of software and into organizational design ought to be taught in every business school.


Work in the Future The Automation Revolution-Palgrave MacMillan (2019) by Robert Skidelsky Nan Craig

3D printing, Airbnb, algorithmic trading, Amazon Web Services, anti-work, artificial general intelligence, autonomous vehicles, basic income, business cycle, cloud computing, collective bargaining, correlation does not imply causation, creative destruction, data is the new oil, David Graeber, David Ricardo: comparative advantage, deindustrialization, deskilling, disintermediation, Donald Trump, Erik Brynjolfsson, feminist movement, Frederick Winslow Taylor, future of work, gig economy, global supply chain, income inequality, informal economy, Internet of things, Jarndyce and Jarndyce, Jarndyce and Jarndyce, job automation, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, John von Neumann, Joseph Schumpeter, knowledge economy, Loebner Prize, low skilled workers, Lyft, Mark Zuckerberg, means of production, moral panic, Network effects, new economy, off grid, pattern recognition, post-work, Ronald Coase, Second Machine Age, self-driving car, sharing economy, Steve Jobs, strong AI, technoutopianism, The Chicago School, The Future of Employment, the market place, The Nature of the Firm, The Wealth of Nations by Adam Smith, Thorstein Veblen, Turing test, Uber for X, uber lyft, universal basic income, wealth creators, working poor

They are what I have elsewhere called a ‘lean platform’.1 They aim to be very asset light: they try to own as little as possible. Uber, for instance does not own the cars; they do not have to pay for fuel; they are not responsible for car insurance or maintenance or anything like that. Even in the core of the business, they do not own massive computer servers or anything. Instead, they rent them out from platforms like Amazon Web Services. Effectively, the Uber model has been to try to own as little as possible. But what they do own is the technological platform that connects riders with passengers, and that is the source of their value extraction. The problem for Uber (and likeminded companies) is that this model has not been very profitable. Lean platforms in general tend to have very low margins. This works for some services, such as things that are very high frequency.

And as companies with extensive data extraction are the most likely to benefit from the virtuous cycle, it means a further consolidation of power, data and resources in the hands of the few companies that already dominate the platform economy. This is all the more important because AI is a general purpose technology.10 AI’s impact will be felt across the economy—so those who control AI, who can actually do AI, are going to have major power and influence in the economy. For instance, for economic impacts we might think about Amazon Web Services or Google Cloud building up AI that they can rent out to other businesses that are too data-poor to build their own AI. They will effectively rent out the basic infrastructure of the digital economy. Likewise, we can imagine companies becoming the sole provider of a particular service. Facebook, for example, already uses such dominance in the attention economy to shape otherwise powerful media companies.


pages: 589 words: 69,193

Mastering Pandas by Femi Anthony

Amazon Web Services, Bayesian statistics, correlation coefficient, correlation does not imply causation, Debian, en.wikipedia.org, Internet of things, natural language processing, p-value, random walk, side project, statistical model, Thomas Bayes

There has been a proliferation of digital data input devices such as cameras and wearables, and the cost of huge data storage has fallen rapidly. Amazon Web Services is a prime example of the trend toward much cheaper storage. The Internetification of devices, or rather Internet of Things, is the phenomenon wherein common household devices, such as our refrigerators and cars, will be connected to the Internet. This phenomenon will only accelerate the above trend. Velocity of big data From a purely technological point of view, velocity refers to the throughput of big data, or how fast the data is coming in and is being processed. This has ramifications on how fast the recipient of the data needs to process it to keep up. Real-time analytics is one attempt to handle this characteristic. Tools that can help enable this include Amazon Web Services Elastic Map Reduce. At a more macro level, the velocity of data can also be regarded as the increased speed at which data and information can now be transferred and processed faster and at greater distances than ever before.


pages: 441 words: 136,954

That Used to Be Us by Thomas L. Friedman, Michael Mandelbaum

addicted to oil, Affordable Care Act / Obamacare, Albert Einstein, Amazon Web Services, American Society of Civil Engineers: Report Card, Andy Kessler, Ayatollah Khomeini, bank run, barriers to entry, Berlin Wall, blue-collar work, Bretton Woods, business process, call centre, carbon footprint, Carmen Reinhart, Cass Sunstein, centre right, Climatic Research Unit, cloud computing, collective bargaining, corporate social responsibility, creative destruction, Credit Default Swap, crowdsourcing, delayed gratification, energy security, Fall of the Berlin Wall, fear of failure, full employment, Google Earth, illegal immigration, immigration reform, income inequality, Intergovernmental Panel on Climate Change (IPCC), job automation, Kenneth Rogoff, knowledge economy, Lean Startup, low skilled workers, Mark Zuckerberg, market design, mass immigration, more computing power than Apollo, Network effects, obamacare, oil shock, pension reform, Report Card for America’s Infrastructure, rising living standards, Ronald Reagan, Rosa Parks, Saturday Night Live, shareholder value, Silicon Valley, Silicon Valley startup, Skype, Steve Jobs, the scientific method, Thomas L Friedman, too big to fail, University of East Anglia, WikiLeaks

The beauty of the cloud, and the reason that it is driving the flattening further and faster, is that it can turn any desktop, laptop, or simple handheld device with a browser into an information-creation or -consumption powerhouse by serving as a central location for those myriad applications, which run on individual user’s devices. Amazon.com, for example, is now selling not only books and chain saws but business facilities in the cloud. Andy Jassy is Amazon’s senior vice president in charge of Amazon Web Services, Bloomberg BusinessWeek explained (March 3, 2011), which means that his job is to rent space to individual innovators, or companies, on Amazon’s rent-a-cloud. Although all shoppers are welcome, this Amazon, [Jassy] explains, is for business customers and isn’t well marked on the home page. It’s called Amazon Web Services, or AWS … , which rents out computing power for pennies an hour. “This completely levels the playing field,” Jassy boasts. AWS makes it possible for anyone with an Internet connection and a credit card to access the same kind of world-class computing systems that Amazon uses to run its $34 billion-a-year retail operation … AWS is growing like crazy.

ACT Adams, Henry Adams, John Advanced Research Projects Agency Afghanistan; Soviet invasion of; U.S. war in African Americans; education of; in World War II Age of Fracture (Rodgers) Ahansal, Mustapha Ahmetovic, Belma Air Force, U.S. Alabama, University of; Creative Campus Alibaba Alice’s Adventures in Wonderland (Carroll) Alito, Samuel Allegheny College Allen, Woody All in the Family (television show) Alonzo, Amanda Alpoge, Levent al-Qaeda Aman, Peter Amanpour, Christiane Amazon; Web Services (AWS) America COMPETES Act (2007) American Association of Retired Persons (AARP) American Federation of State, County and Municipal Employees American Federation of Teachers American Interest, The Americans and the California Dream (Starr) American Society of Civil Engineers (ASCE) American Solutions American Telephone and Telegraph (AT&T) Amtrak Acela Anand, Namrata Andersen, Kurt Anderson, Chris Android Angelides, Phil anti-Federalists AOL Apollo space program Apotheker, Léo Apple; iPad; iPhone; iPod; Macintosh computers Applied Materials apps Arab oil embargo Arab world, uprisings in Argonne National Laboratory Arkansas Armey, Dick Army, U.S.; Training and Doctrine Command Asato, Cathy Asia Society, Center on U.S.


pages: 371 words: 78,103

Webbots, Spiders, and Screen Scrapers by Michael Schrenk

Amazon Web Services, corporate governance, fault tolerance, Firefox, Marc Andreessen, new economy, pre–internet, SpamAssassin, The Hackers Conference, Turing test, web application

An example of a SOAP call is shown in Listing 26-13. In typical SOAP calls, the SOAP interface and client are created and the parameters describing requested web services are passed in an array. With SOAP, using a web service is much like calling a local function. If you'd like to experiment with SOAP, consider creating a free account at Amazon Web Services. Amazon provides SOAP interfaces that allow you to access large volumes of data at both Amazon and Alexa, a web-monitoring service (http://www.alexa.com). Along with Amazon Web Services, you should also review the PHP-specific Amazon SOAP tutorial at Dev Shed, a PHP developers' site (http://www.devshed.com). PHP 5 has built-in support for SOAP. If you're using PHP 4, however, you will need to use the appropriate PHP Extension and Application Repository (PEAR, http://www.pear.php.net) libraries, included in most PHP distributions.


pages: 252 words: 78,780

Lab Rats: How Silicon Valley Made Work Miserable for the Rest of Us by Dan Lyons

Airbnb, Amazon Web Services, Apple II, augmented reality, autonomous vehicles, basic income, bitcoin, blockchain, business process, call centre, Clayton Christensen, clean water, collective bargaining, corporate governance, corporate social responsibility, creative destruction, cryptocurrency, David Heinemeier Hansson, Donald Trump, Elon Musk, Ethereum, ethereum blockchain, full employment, future of work, gig economy, Gordon Gekko, greed is good, hiring and firing, housing crisis, income inequality, informal economy, Jeff Bezos, job automation, job satisfaction, job-hopping, John Gruber, Joseph Schumpeter, Kevin Kelly, knowledge worker, Lean Startup, loose coupling, Lyft, Marc Andreessen, Mark Zuckerberg, McMansion, Menlo Park, Milgram experiment, minimum viable product, Mitch Kapor, move fast and break things, move fast and break things, new economy, Panopticon Jeremy Bentham, Paul Graham, paypal mafia, Peter Thiel, plutocrats, Plutocrats, precariat, RAND corporation, remote working, RFID, ride hailing / ride sharing, Ronald Reagan, Rubik’s Cube, Ruby on Rails, Sam Altman, Sand Hill Road, self-driving car, shareholder value, Silicon Valley, Silicon Valley startup, six sigma, Skype, Social Responsibility of Business Is to Increase Its Profits, software is eating the world, Stanford prison experiment, stem cell, Steve Jobs, Steve Wozniak, Stewart Brand, TaskRabbit, telemarketer, Tesla Model S, Thomas Davenport, Tony Hsieh, Toyota Production System, traveling salesman, Travis Kalanick, tulip mania, Uber and Lyft, Uber for X, uber lyft, universal basic income, web application, Whole Earth Catalog, Y Combinator, young professional

To distinguish between its two businesses, Q labels its original cleaning business Q Services. That group—the cleaning business—still generates 95 percent of Q’s revenues, but eventually Teran thinks Q Marketplace could become the larger part of the business. Teran’s ultimate goal is to handle every aspect of managing a physical space. He uses Amazon Web Services as a model. Instead of building and managing their own data centers, today most companies just fill out a few forms and rent computer power from Amazon Web Services. They might have no idea what kind of computers are being used, or who is running the servers. In the ultimate version of this, a law firm or ad agency could rent new offices, call Q, and never worry about the physical space again. Q would do everything and send you a single bill every month. Some chores, like cleaning, would be done by Q employees.


pages: 305 words: 79,303

The Four: How Amazon, Apple, Facebook, and Google Divided and Conquered the World by Scott Galloway

activist fund / activist shareholder / activist investor, additive manufacturing, Affordable Care Act / Obamacare, Airbnb, Amazon Web Services, Apple II, autonomous vehicles, barriers to entry, Ben Horowitz, Bernie Sanders, big-box store, Bob Noyce, Brewster Kahle, business intelligence, California gold rush, cloud computing, commoditize, cuban missile crisis, David Brooks, disintermediation, don't be evil, Donald Trump, Elon Musk, follow your passion, future of journalism, future of work, global supply chain, Google Earth, Google Glasses, Google X / Alphabet X, Internet Archive, invisible hand, Jeff Bezos, Jony Ive, Khan Academy, longitudinal study, Lyft, Mark Zuckerberg, meta analysis, meta-analysis, Network effects, new economy, obamacare, Oculus Rift, offshore financial centre, passive income, Peter Thiel, profit motive, race to the bottom, RAND corporation, ride hailing / ride sharing, risk tolerance, Robert Mercer, Robert Shiller, Robert Shiller, Search for Extraterrestrial Intelligence, self-driving car, sentiment analysis, shareholder value, Silicon Valley, Snapchat, software is eating the world, speech recognition, Stephen Hawking, Steve Ballmer, Steve Jobs, Steve Wozniak, Stewart Brand, supercomputer in your pocket, Tesla Model S, Tim Cook: Apple, Travis Kalanick, Uber and Lyft, Uber for X, uber lyft, undersea cable, Whole Earth Catalog, winner-take-all economy, working poor, young professional

In the past few years, Amazon has traded on this brand equity, leveraging it to extend into other businesses, and has expanded into other, simply better (more profitable) businesses. Looking back, Amazon’s retail platform just may have been the Trojan Horse that established the relationships and brand later monetized with other businesses. While year-to-year growth for Amazon’s retail business ranged from 13 percent to 20 percent from Q1 to Q3 2015, Amazon Web Services—the retailer’s network of servers and data storage technology—has grown 49 percent to 81 percent during that same interval. AWS also grew into a significant portion of Amazon’s total operating income, from 38 percent in Q1 2015 to 52 percent in Q3 2015.60 Analysts predict that AWS could reach $16.2 billion in sales by the end of 2017, making it worth $160 billion—more than the company’s retail unit.61 In other words, while the world still thinks of Amazon as a retailer, it has quietly become a cloud company—the world’s biggest.

You may need to scroll forward from that location to find the corresponding reference on your e-reader. Note: Page numbers in italic refer to illustrations. About.com, 147–48, 150–51 Age of Enlightenment, 127 Airbnb, 225–27 Alibaba, 206, 206–10 Amazon, 13–62 agility of, 235 and Alexa/Echo, 9, 31, 45, 50–52, 56 algorithm of, 105, 106 Amazon Marketplace, 25, 41 Amazon Prime, 13, 14, 37, 42, 48, 48–49 Amazon Web Services, 37, 41–42 analog moats of, 91 automated warehouses of, 27, 29–30, 42, 52–54, 91 book sales of, 25–26, 45 brain-based appeal of, 176–77 and brand era, 50–52, 172–73 capitalization of, 3, 32–35, 36, 38–39, 54, 56, 188–89 and competition, 9, 25, 56, 181 consumers’ alliance with, 167–68 consumer trust in, 31, 32, 41, 56 current state of affairs, 3–4 data collection of, 32, 200 effect on retail sector, 24–25, 28–29, 57–59 and failure, 39, 40 frictionless purchases on, 32, 186–87 fulfillment infrastructure, 3, 42, 49, 54, 55, 56, 91, 166, 181, 186 future of, 165–66, 182 global reach of, 190–91, 191 and Google, 60, 188–89 growth of, 27–28, 34, 41, 53 headquarters of, 202 and hunter/gatherer strategies, 14–16, 25–26, 44, 176–77 likability of, 193 multichannel retail, 44–49 and Procter & Gamble (P&G), 194 product differentiation of, 186–87 and product searches, 8, 189 profits/revenue, 17–18, 34–35 and Quidsi, 46 and rational decision making, 170 risk taking, 35–39, 40 scalability, 24–25 sellers’ joining of, 166 stock appreciation of, 28, 28 and stores, 30, 31, 36, 43–45, 55, 56 and taxes, 34, 193 vertical integration of, 195 and Walmart, 228 Whole Foods acquisition, 44, 54, 55–56, 59 zero-click ordering, 31–32, 56 See also Bezos, Jeff analog moats, 10, 90–91, 92 Android, 155, 178 Apollo Project, 267 Apple, 63–95 algorithms of, 105 analog moats of, 90–91, 92 behavioral data of, 200 and competition, 9, 67, 90–91, 92, 179–81 computers, 69, 73–74, 132, 161, 180 Cook’s leadership of, 68–69, 87 current state of affairs, 4 and download piracy, 68 and education market, 93–95 and failure, 39 genitalia-based appeal of, 178–79 global reach of, 190, 191 headquarters of, 202 innovator status, 64, 67, 68, 161 intellectual property theft, 161, 166 iPod, 67–68, 74, 79 likability of, 193 low-cost production of, 88–89 luxury status of, 69–72, 74–76, 78–79, 80–86, 87, 88, 92 privacy issue, 63–64, 136 product differentiation of, 186 profits/revenue, 4, 69, 70, 85 risk taking, 68 secular worship of, 66–67 stores, 80–81, 82–83, 89, 91, 195–96 strategy for success, 57 and taxes, 83, 193, 197 vertical integration of, 195–96 vs.


pages: 339 words: 88,732

The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies by Erik Brynjolfsson, Andrew McAfee

"Robert Solow", 2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, 3D printing, access to a mobile phone, additive manufacturing, Airbnb, Albert Einstein, Amazon Mechanical Turk, Amazon Web Services, American Society of Civil Engineers: Report Card, Any sufficiently advanced technology is indistinguishable from magic, autonomous vehicles, barriers to entry, basic income, Baxter: Rethink Robotics, British Empire, business cycle, business intelligence, business process, call centre, Charles Lindbergh, Chuck Templeton: OpenTable:, clean water, combinatorial explosion, computer age, computer vision, congestion charging, corporate governance, creative destruction, crowdsourcing, David Ricardo: comparative advantage, digital map, employer provided health coverage, en.wikipedia.org, Erik Brynjolfsson, factory automation, falling living standards, Filter Bubble, first square of the chessboard / second half of the chessboard, Frank Levy and Richard Murnane: The New Division of Labor, Freestyle chess, full employment, G4S, game design, global village, happiness index / gross national happiness, illegal immigration, immigration reform, income inequality, income per capita, indoor plumbing, industrial robot, informal economy, intangible asset, inventory management, James Watt: steam engine, Jeff Bezos, jimmy wales, job automation, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, Joseph Schumpeter, Kevin Kelly, Khan Academy, knowledge worker, Kodak vs Instagram, law of one price, low skilled workers, Lyft, Mahatma Gandhi, manufacturing employment, Marc Andreessen, Mark Zuckerberg, Mars Rover, mass immigration, means of production, Narrative Science, Nate Silver, natural language processing, Network effects, new economy, New Urbanism, Nicholas Carr, Occupy movement, oil shale / tar sands, oil shock, pattern recognition, Paul Samuelson, payday loans, post-work, price stability, Productivity paradox, profit maximization, Ralph Nader, Ray Kurzweil, recommendation engine, Report Card for America’s Infrastructure, Robert Gordon, Rodney Brooks, Ronald Reagan, Second Machine Age, self-driving car, sharing economy, Silicon Valley, Simon Kuznets, six sigma, Skype, software patent, sovereign wealth fund, speech recognition, statistical model, Steve Jobs, Steven Pinker, Stuxnet, supply-chain management, TaskRabbit, technological singularity, telepresence, The Bell Curve by Richard Herrnstein and Charles Murray, The Signal and the Noise by Nate Silver, The Wealth of Nations by Adam Smith, total factor productivity, transaction costs, Tyler Cowen: Great Stagnation, Vernor Vinge, Watson beat the top human players on Jeopardy!, winner-take-all economy, Y2K

Today, people with connected smartphones or tablets anywhere in the world have access to many (if not most) of the same communication resources and information that we do while sitting in our offices at MIT. They can search the Web and browse Wikipedia. They can follow online courses, some of them taught by the best in the academic world. They can share their insights on blogs, Facebook, Twitter, and many other services, most of which are free. They can even conduct sophisticated data analyses using cloud resources such as Amazon Web Services and R, an open source application for statistics.13 In short, they can be full contributors in the work of innovation and knowledge creation, taking advantage of what Autodesk CEO Carl Bass calls “infinite computing.”14 Until quite recently rapid communication, information acquisition, and knowledge sharing, especially over long distances, were essentially limited to the planet’s elite. Now they’re much more democratic and egalitarian, and getting more so all the time.

id=USARGDPH INDEX Academically Adrift: Limited Learning on College Campuses (Arum and Roksa) Acemoglu, Daron Affinnova Aftercollege.com Agarwal, Anant Age of Spiritual Machines, The: When Computers Exceed Human Intelligence (Kurzweil) Agrarian Justice (Paine) agriculture: development of inelastic demand in Ahn, Luis von Aiden, Erez Lieberman Airbnb.com Alaska, income guarantee plan in algorithms Allegretto, Sylvia Allstate Amazon Amazon Web Services American Society of Civil Engineers (ASCE) Android animals, domestication of Apple Arthur, Brian artificial intelligence (AI) future of SLAM problem in uses of see also robots Arum, Richard ASCI Red ASIMO Asimov, Isaac Asur, Sitaram Athens, ancient ATMs Audi Australia, immigrant entrepreneurship in Autodesk automation: future of labor market effects of in manufacturing Autor, David Baker, Stephen Barnes & Noble Bartlett, Albert A.


pages: 329 words: 95,309

Digital Bank: Strategies for Launching or Becoming a Digital Bank by Chris Skinner

algorithmic trading, AltaVista, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, augmented reality, bank run, Basel III, bitcoin, business cycle, business intelligence, business process, business process outsourcing, buy and hold, call centre, cashless society, clean water, cloud computing, corporate social responsibility, credit crunch, crowdsourcing, cryptocurrency, demand response, disintermediation, don't be evil, en.wikipedia.org, fault tolerance, fiat currency, financial innovation, Google Glasses, high net worth, informal economy, Infrastructure as a Service, Internet of things, Jeff Bezos, Kevin Kelly, Kickstarter, M-Pesa, margin call, mass affluent, MITM: man-in-the-middle, mobile money, Mohammed Bouazizi, new economy, Northern Rock, Occupy movement, Pingit, platform as a service, Ponzi scheme, prediction markets, pre–internet, QR code, quantitative easing, ransomware, reserve currency, RFID, Satoshi Nakamoto, Silicon Valley, smart cities, social intelligence, software as a service, Steve Jobs, strong AI, Stuxnet, trade route, unbanked and underbanked, underbanked, upwardly mobile, We are the 99%, web application, WikiLeaks, Y2K

By doing this, Amazon built its business on finding offers that you might buy, because people like you buy it, and they know this thanks to our unique dataprints. Soon, Amazon was more of a behemoth of data, moving into selling anything from white goods to televisions, and it is easy to sell online when you know how to leverage data relationships. Even then, for Amazon, even this was not enough. In recognising its information leadership, Amazon opened Amazon Web Services (AWS) to become the largest cloud computing firm out there. Amazon now adds server systems to AWS every day that would have been the equivalent of the complete server architecture required to run the total retail business two years ago. That’s information warfare: leveraging systems expertise to get more share of wallet, expansion of market, growth of proposition, and development of the offer into a range of services with no dependency on one.

The Innotribe Start-up Challenge was designed to introduce the most promising FinTech and Financial Services Start-Ups to SWIFT’s community of more than 9,700 banking organisations, securities institutions and corporate customers in 209 countries. THE CURRENCY CLOUD, GLOBAL An interview with Michael Laven, Chief Executive, the Currency Cloud Cloud computing has been around for a while, but still suffers from some extreme views with some considering cloud to just be Amazon Web Services whilst others see this as a way of leveraging new forms of business models. Michael (Mike) Laven, CEO of Currency Cloud is one of the latter visionaries, and is changing the game in cross-border payments by building a cloud-based offer for currency movements and offering this as a low-cost, transparent service to consumers and small businesses. What is the Currency Cloud? There is a huge volume of cross-border transactions that are well served for the large, multinational firms, but a major, untapped and underserved market for the cross-border payments needs of smaller firms.


pages: 328 words: 96,141

Rocket Billionaires: Elon Musk, Jeff Bezos, and the New Space Race by Tim Fernholz

Amazon Web Services, autonomous vehicles, business climate, Charles Lindbergh, Clayton Christensen, cloud computing, Colonization of Mars, corporate governance, corporate social responsibility, disruptive innovation, Donald Trump, Elon Musk, high net worth, Iridium satellite, Jeff Bezos, Kickstarter, low earth orbit, Marc Andreessen, Mark Zuckerberg, minimum viable product, multiplanetary species, mutually assured destruction, new economy, nuclear paranoia, paypal mafia, Peter H. Diamandis: Planetary Resources, Peter Thiel, pets.com, planetary scale, private space industry, profit maximization, RAND corporation, Richard Feynman, Richard Feynman: Challenger O-ring, Ronald Reagan, shareholder value, Silicon Valley, skunkworks, sovereign wealth fund, Stephen Hawking, Steve Jobs, trade route, undersea cable, We wanted flying cars, instead we got 140 characters, X Prize, Y2K

Bezos told Simpson that he was building a spaceport. News traveled around the country quickly. Bezos declined to speak to the AP about the project, and a spokesman simply said that the company “won’t go anywhere soon.” That, at least, was true. Amazon was already big, but not yet the Goliath it is today. This was before iPhones led the smartphone revolution, before the Kindle, before Amazon Prime, and before Amazon Web Services and Alexa. At this stage in its lifespan, the retailer hadn’t even made an annual profit, despite spending the past four years as a publicly traded company. Investors loved the stock because of its incredible growth rates, the way it scarfed up market share, even entire markets, with ravenous energy. The idea of Bezos devoting time and energy to another company, especially one with such a nebulous future, wasn’t likely to play well with investors who already tolerated quite a bit of eccentricity.

It was a period of huge expansion for the online company, as it went from national retailer to global force. In the years ahead, Amazon would begin releasing physical products like the Kindle, a short-lived phone, and smart speakers for its AI assistant. It would be an early adopter of the growth in cloud computing and deploy its own proprietary servers, optimized to allow other digital entrepreneurs to scale up their own services. Amazon Web Services became a huge moneymaker that undergirded a new generation of internet companies. Amazon established its own distribution centers and began investing in robotic technology to automate deliveries, even experimenting with drones for airborne package drop-offs. This period may have been a distraction from the work of making space cheaper to access, but it would pay off in literal dividends in the years ahead.


pages: 571 words: 105,054

Advances in Financial Machine Learning by Marcos Lopez de Prado

algorithmic trading, Amazon Web Services, asset allocation, backtesting, bioinformatics, Brownian motion, business process, Claude Shannon: information theory, cloud computing, complexity theory, correlation coefficient, correlation does not imply causation, diversification, diversified portfolio, en.wikipedia.org, fixed income, Flash crash, G4S, implied volatility, information asymmetry, latency arbitrage, margin call, market fragmentation, market microstructure, martingale, NP-complete, P = NP, p-value, paper trading, pattern recognition, performance metric, profit maximization, quantitative trading / quantitative finance, RAND corporation, random walk, risk-adjusted returns, risk/return, selection bias, Sharpe ratio, short selling, Silicon Valley, smart cities, smart meter, statistical arbitrage, statistical model, stochastic process, survivorship bias, transaction costs, traveling salesman

Tolle (2009): The Fourth Paradigm: Data-Intensive Scientific Discovery. Vol. 1. Microsoft research Redmond, WA. Hirschman, A. O. (1980): National Power and the Structure of Foreign Trade. Vol. 105. University of California Press. Holzman, B. et al. (2017): “HEPCloud, a new paradigm for HEP facilities: CMS Amazon Web Services investigation. Computing and Software for Big Science, Vol. 1, No. 1, p. 1. Jackson, K. R., et al. (2010): “Performance analysis of high performance computing applications on the Amazon Web Services Cloud. Cloud Computing Technology and Science (CloudCom). 2010 Second International Conference. IEEE. Kim, T. et al. (2015): “Extracting baseline electricity usage using gradient tree boosting.” IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity). IEEE. Kumar, V. et al. (1994): Introduction to Parallel Computing: Design and Analysis of Algorithms.


pages: 349 words: 95,972

Messy: The Power of Disorder to Transform Our Lives by Tim Harford

affirmative action, Air France Flight 447, Airbnb, airport security, Albert Einstein, Amazon Mechanical Turk, Amazon Web Services, assortative mating, Atul Gawande, autonomous vehicles, banking crisis, Barry Marshall: ulcers, Basel III, Berlin Wall, British Empire, Broken windows theory, call centre, Cass Sunstein, Chris Urmson, cloud computing, collateralized debt obligation, crowdsourcing, deindustrialization, Donald Trump, Erdős number, experimental subject, Ferguson, Missouri, Filter Bubble, Frank Gehry, game design, global supply chain, Googley, Guggenheim Bilbao, high net worth, Inbox Zero, income inequality, industrial cluster, Internet of things, Jane Jacobs, Jeff Bezos, Loebner Prize, Louis Pasteur, Marc Andreessen, Mark Zuckerberg, Menlo Park, Merlin Mann, microbiome, out of africa, Paul Erdős, Richard Thaler, Rosa Parks, self-driving car, side project, Silicon Valley, Silicon Valley startup, Skype, Steve Jobs, Steven Levy, Stewart Brand, telemarketer, the built environment, The Death and Life of Great American Cities, Turing test, urban decay, William Langewiesche

Instead, over the next few years, Amazon launched products as disparate as the Kindle (which immediately and repeatedly sold out, as Amazon struggled to manufacture it), Mechanical Turk (an unsettlingly named global clearinghouse for labor, which pioneered crowdsourcing but was criticized as being a sweatshop), the Fire Phone (widely reviewed as ugly, weird, and disappointing), Marketplace (where competitors to Amazon would use Amazon’s own product listings to advertise their own cheaper alternatives), and Amazon Web Services. AWS in particular was a bold stroke—a move into cloud computing in 2006, four years ahead of Microsoft’s Azure and six years ahead of Google Compute. As Bezos liked to say during the crunches of 1998 and 1999, “If you are planning more than twenty minutes ahead in this environment, you are wasting your time.”15 He was a man in a hurry. No wonder he created such an almighty mess. • • • Come the Second World War, the Italians were the Germans’ allies, but they seemed as prone as ever to losing battles.

By Christmas 2009, Amazon had a market share in e-books of around 90 percent.34 One can tell almost exactly the same story about Amazon’s initially baffling venture into cloud computing: a crazy rush, a series of early technical problems, a loss-making price. Then within a few years, Amazon, a mere bookseller, was the dominant player in cloud computing, and analysts were pronouncing that Amazon Web Services was a more valuable business than Amazon’s online retailing operation. The opportunity had been there to take, but the titans of the industry, IBM, Google, Apple, and Microsoft, had all hesitated at the prospect of a costly battle with the upstart.35 Again and again, we see Amazon moving quickly, losing money, struggling to cope with the demand it created, and in the end, dominating a market.


pages: 161 words: 30,412

Creating Development Environments With Vagrant - Second Edition by Michael Peacock

Amazon Web Services, cloud computing, continuous integration, Debian, domain-specific language, web application

This can involve syncing folders such that the project code, which you edit using the IDE on your computer, is synced so that it runs on the Vagrant development environment. Vagrant uses providers to integrate with the third-party virtualization software, which provides the virtualized machines for our development environment. The default provider is for Oracle VirtualBox; however, there are commercial providers to work with VMware Fusion and also plugins for Vagrant to work with Amazon Web Services. The entire configuration is stored in simple plain text files. The Vagrant configuration (Vagrantfile), and the configuration that defines how our Vagrant machines are configured (typically Shell scripts, Ansible playbooks, Chef cookbooks or Puppet manifests that Vagrant has built-in support for, as provisioners) are simply written in text files. This means that we can easily share the configurations and projects with colleagues, using version control systems such as Git.


pages: 903 words: 235,753

The Stack: On Software and Sovereignty by Benjamin H. Bratton

1960s counterculture, 3D printing, 4chan, Ada Lovelace, additive manufacturing, airport security, Alan Turing: On Computable Numbers, with an Application to the Entscheidungsproblem, algorithmic trading, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, basic income, Benevolent Dictator For Life (BDFL), Berlin Wall, bioinformatics, bitcoin, blockchain, Buckminster Fuller, Burning Man, call centre, carbon footprint, carbon-based life, Cass Sunstein, Celebration, Florida, charter city, clean water, cloud computing, connected car, corporate governance, crowdsourcing, cryptocurrency, dark matter, David Graeber, deglobalization, dematerialisation, disintermediation, distributed generation, don't be evil, Douglas Engelbart, Douglas Engelbart, Edward Snowden, Elon Musk, en.wikipedia.org, Eratosthenes, Ethereum, ethereum blockchain, facts on the ground, Flash crash, Frank Gehry, Frederick Winslow Taylor, future of work, Georg Cantor, gig economy, global supply chain, Google Earth, Google Glasses, Guggenheim Bilbao, High speed trading, Hyperloop, illegal immigration, industrial robot, information retrieval, Intergovernmental Panel on Climate Change (IPCC), intermodal, Internet of things, invisible hand, Jacob Appelbaum, Jaron Lanier, Joan Didion, John Markoff, Joi Ito, Jony Ive, Julian Assange, Khan Academy, liberal capitalism, lifelogging, linked data, Mark Zuckerberg, market fundamentalism, Marshall McLuhan, Masdar, McMansion, means of production, megacity, megastructure, Menlo Park, Minecraft, MITM: man-in-the-middle, Monroe Doctrine, Network effects, new economy, offshore financial centre, oil shale / tar sands, packet switching, PageRank, pattern recognition, peak oil, peer-to-peer, performance metric, personalized medicine, Peter Eisenman, Peter Thiel, phenotype, Philip Mirowski, Pierre-Simon Laplace, place-making, planetary scale, RAND corporation, recommendation engine, reserve currency, RFID, Robert Bork, Sand Hill Road, self-driving car, semantic web, sharing economy, Silicon Valley, Silicon Valley ideology, Slavoj Žižek, smart cities, smart grid, smart meter, social graph, software studies, South China Sea, sovereign wealth fund, special economic zone, spectrum auction, Startup school, statistical arbitrage, Steve Jobs, Steven Levy, Stewart Brand, Stuxnet, Superbowl ad, supply-chain management, supply-chain management software, TaskRabbit, the built environment, The Chicago School, the scientific method, Torches of Freedom, transaction costs, Turing complete, Turing machine, Turing test, undersea cable, universal basic income, urban planning, Vernor Vinge, Washington Consensus, web application, Westphalian system, WikiLeaks, working poor, Y Combinator

It is at this level of The Stack that the modern coherence of the state, which would produce one sort of public, and the operations of platforms, which would produce another, can come into conflict, overlapping and interlacing one another without universal jurisdiction or resolution, but it is also where they can reinforce each other with more pervasive forms of ambient governance. The geopolitics of the Cloud is everywhere and wants everything: the platform wars between Google, Facebook, Apple, and Amazon, anonymized servers routing the angry tweets from street battles, Anonymous going up against Mexican drug cartels, WikiLeaks crowd-sourcing counterespionage, Tor users building on top of Amazon Web Services services, carriers licensing content, content providers licensing bandwidth, proprietary fiber networks connected trading centers, and on, and on. It might seem at first blush that these events, each perhaps pushing legal boundaries in its own way, should be understood as disruptive contaminations of a standing political order—acts of resistance to the system, even. Yet in their own consistency, this stockpile of exceptions is probably better interpreted as part of the constitution of another emergent order (a nomos of the Cloud even?).

That said, as the Cloud is planetary is scope, state control of its systems is guaranteed only to the extent that private providers continue to respect (by consent, force, joint venture, outright merger) the practical sovereignty of the national jurisdictions in which their servers are installed, where their offices are headquartered, how exactly their data structure economic exchanges, and how their monetization of those data is or is not taxed. That arrangement is both resilient and unstable in various measures and tracks the successes and failures of globalization itself. Today most Cloud service providers have constraining jurisdictions built into service plans. For example, Amazon Web Services segregates the serving of hosted data according to several geographic “availability zones,” allowing developers to deploy specific versions of their application to specific users in specific countries according to local laws and priorities, regardless of where Amazon might be hosting or mirroring their data. However, the sorting of state space and data space is not always so neat, and the unintended effects of innovative interconnections between Cloud publics can be calamitous.

The platform does not care about your name or who you really are deep down, but only in the likelihood that the next presentation of object X, Y, or Z will motivate your One-Clicking and the subsequent activation of supply chain cascades that ensue. For this, Amazon does not even necessarily have to provide the User-facing front end for Cloud services. Just as UPS and FedEx moved into high-margin logistics consulting businesses, Amazon Web Services is a major provider of large-scale Cloud hosting and e-commerce for third parties (including states and their intelligence agencies). In addition to finding customers for its suppliers, Amazon also rents the pick-axes for the Cloud rush. This superterranean platform-of-platforms (from servers to warehouse to inventory delivery) allows both small and large affiliate actors to engage multiple overlapping and even opposed publics on the same shared hosting infrastructures.


pages: 178 words: 33,275

Ansible Playbook Essentials by Gourav Shah

Amazon Web Services, cloud computing, Debian, DevOps, fault tolerance, web application

Before using this subcommand, it is important to set an editor in the environment, as follows: # setting up vi as editor $ export EDITOR=vi # Generate a encrypted file $ ansible-vault create aws_creds.yml Vault password: Confirm Vault password: Launching this command opens up an editor specified with the editor environment variable. The following is an example of the aws_creds.yml file that you may create to store the AWS user credentials in the form of an access key and secret key. These keys are then used to make API calls to the Amazon web services cloud platform. Saving this file and exiting the editor will generate an encrypted file: You can check the type of file created and its contents by running following commands: # Check file type and content $ file aws_creds.yml aws_creds.yml: ASCII text $ cat aws_creds.yml $ANSIBLE_VAULT;1.1;AES256 64616236666362376630366435623538336565393331333331663663636237636335313234313134 3337303865323239623436646630336239653864356561640a363966393135316661636562333932 61323932313230383433313735646438623032613635623966646232306433383335326566343333 3136646536316261300a616438643463656263636237316136356163646161313365336239653434 36626135313138343939363635353563373865306266363532386537623463623464376134353863 37646638636231303461343564343232343837356662316262356537653066356465353432396436 31336664313661306630653765356161616266653232316637653132356661343162396331353863 34356632373963663230373866313961386435663463656561373461623830656261636564313464 37383465353665623830623363353161363033613064343932663432653666633538 Updating the encrypted data To update the AWS keys added to the encrypted file, you can later use Ansible-vault's edit subcommand as follows: $ ansible-vault edit aws_creds.yml Vault password: The edit command does the following operations: Prompts for a password Decrypts a file on the fly using the AES symmetric cypher Opens the editor interface, which allows you to change the content of a file Encrypts the file again after being saved There is another way to update the content of the file.


pages: 757 words: 193,541

The Practice of Cloud System Administration: DevOps and SRE Practices for Web Services, Volume 2 by Thomas A. Limoncelli, Strata R. Chalup, Christina J. Hogan

active measures, Amazon Web Services, anti-pattern, barriers to entry, business process, cloud computing, commoditize, continuous integration, correlation coefficient, database schema, Debian, defense in depth, delayed gratification, DevOps, domain-specific language, en.wikipedia.org, fault tolerance, finite state, Firefox, Google Glasses, information asymmetry, Infrastructure as a Service, intermodal, Internet of things, job automation, job satisfaction, Kickstarter, load shedding, longitudinal study, loose coupling, Malcom McLean invented shipping containers, Marc Andreessen, place-making, platform as a service, premature optimization, recommendation engine, revision control, risk tolerance, side project, Silicon Valley, software as a service, sorting algorithm, standardized shipping container, statistical model, Steven Levy, supply-chain management, Toyota Production System, web application, Yogi Berra

In Linux and the Xen hyper-visor, this is called “steal time”: it is the amount of CPU time that your virtual machine is missing because it was allocated to other virtual machines (Haynes 2013). IaaS providers usually cannot provide guarantees of how much steal time will exist, nor can they provide mechanisms to control it. Netflix found the only way it could deal with this issue was to be reactionary. If high steal time was detected on a virtual machine in Amazon Web Services (AWS), Netflix would delete the virtual machine and have it re-created. If the company was lucky, the new virtual machine would be created on a physical machine that was less oversubscribed. This is a sorry state of affairs (Link 2013). Some resources are shared in an unbounded manner. For example, if one virtual machine is generating a huge amount of network traffic, the other virtual machines may suffer.

., 137 Access Control List (ACL) mechanisms description, 40 Google, 41 Access controls in design for operations, 40–41 Account creation automation example, 251–252 Accuracy, automation for, 253 ACID databases, 24 Acknowledgments for alert messages, 355–356 ACL (Access Control List) mechanisms description, 40 Google, 41 Active-active pairs, 126 Active users, 366 Adaptive Replacement Cache (ARC) algorithm, 107 Adopting design documents, 282–283 Advertising systems in second web era, 465 After-hours oncall maintenance coordination, 294 Agents in collections, 352 Aggregators, 352 Agile Alliance, 189 Agile Manifesto, 189 Agile techniques, 180 continuous delivery, 188–189 feature requests, 264 AKF Scaling Cube, 99 combinations, 104 functional and service splits, 101–102 horizontal duplication, 99–101 lookup-oriented splits, 102–104 Alerts, 163, 285 alerting and escalation systems, 345, 354–357 monitoring, 333 oncall for. See Oncall rules, 353 thresholds, 49 Alexander, Christopher, 69 Allen, Woody, 285 Allspaw, John automation, 249 disaster preparedness tests, 318–320 outage factors, 302 postmortems, 301 Alternatives in design documents, 278 Amazon design process, 276 Game Day, 318 Simple Queue Service, 85 Amazon Elastic Compute Cloud (Amazon EC2), 472 Amazon Web Services (AWS), 59 Analysis in capacity planning, 375–376 causal, 301–302 crash reports, 129 in monitoring, 345, 353–354 Ancillary resources in capacity planning, 372 Andreessen, Marc, 181 Anomaly detection, 354 “Antifragile Organization” article, 315, 320 Antifragile systems, 308–310 Apache systems Hadoop, 132, 467 Mesos, 34 web server forking, 114 Zookeeper, 231, 363 API (Application Programming Interface) defined, 10 logs, 340 Applicability in dot-bomb era, 463–464 Application architectures, 69 cloud-scale service, 80–85 exercises, 93 four-tier web service, 77–80 message bus, 85–90 reverse proxy service, 80 service-oriented, 90–92 single-machine web servers, 70–71 summary, 92–93 three-tier web service, 71–77 Application debug logs, 340 Application logs, 340 Application Programming Interface (API) defined, 10 logs, 340 Application servers in four-tier web service, 79 Approvals code, 47–48 deployment phase, 214, 216–217 design documents, 277, 281 service launches, 159 Arbitrary groups, segmentation by, 104 ARC (Adaptive Replacement Cache) algorithm, 107 Architecture factors in service launches, 157 Archives design documents, 279–280 email, 277 Art of Scalability, Scalable Web Architecture, Processes, and Organizations for the Modern Enterprise, 100 Artifacts artifact-scripted database changes, 185 defined, 196 Assessments, 421–422 Capacity Planning, 431–432 Change Management, 433–434 Disaster Preparedness, 448–450 Emergency Response, 426–428 levels, 405–407 methodology, 403–407 Monitoring and Metrics, 428–430 New Product Introduction and Removal, 435–436 operational excellence, 405–407 organizational, 411–412 Performance and Efficiency, 439–441 questions, 407 Regular Tasks, 423–425 Service Delivery: The Build Phase, 442–443 Service Delivery: The Deployment Phase, 444–445 Service Deployment and Decommissioning, 437–438 services, 407–410 Toil Reduction, 446–447 Asynchronous design, 29 Atlassian Bamboo tool, 205 Atomicity ACID term, 24 release, 240–241 Attack surface area, 79 Auditing operations design, 42–43 Augmentation files, 41–42 Authentication, authorization, and accounting (AAA), 222 Authentication in deployment phase, 222 Authors in design documents, 277, 282 Auto manufacturing automation example, 251 Automation, 243–244 approaches, 244–245 baking, 219 benefits, 154 code amount, 269–270 code reviews, 268–269 compensatory principle, 246–247 complementarity principle, 247–248 continuous delivery, 190 crash data collection and analysis, 129 creating, 255–258 DevOps, 182, 185–186 exercises, 272–273 goals, 252–254 hidden costs, 250 infrastructure strategies, 217–220 issue tracking systems, 263–265 language tools, 258–262 left-over principle, 245–246 lessons learned, 249–250 multitenant systems, 270–271 prioritizing, 257–258 repair life cycle, 254–255 software engineering tools and techniques, 262–270 software packaging, 266 software restarts and escalation, 128–129 steps, 258 style guides, 266–267, 270 summary, 271–272 system administration, 248–249 tasks, 153–155 test-driven development, 267–268 toil reduction, 257 vs. tool building, 250–252 version control systems, 265–266 Availability CAP Principle, 21–22 monitoring, 336 Availability and partition tolerance (AP), 24 Availability requirements cloud computing era, 469 dot-bomb era, 460 first web era, 455 pre-web era, 452–453 second web era, 465 Averages in monitoring, 358 AWS (Amazon Web Services), 59 Backend replicas, load balancers with, 12–13 Backends multiple, 14–15 server stability, 336 Background in design documents, 277–278 Background processes for containers, 61 Backups in design for operations, 36 Baked images for OS installation, 219–220 Banned query lists, 130 Bare metal clouds, 68 Barroso, L.

See Oncall rules, 353 thresholds, 49 Alexander, Christopher, 69 Allen, Woody, 285 Allspaw, John automation, 249 disaster preparedness tests, 318–320 outage factors, 302 postmortems, 301 Alternatives in design documents, 278 Amazon design process, 276 Game Day, 318 Simple Queue Service, 85 Amazon Elastic Compute Cloud (Amazon EC2), 472 Amazon Web Services (AWS), 59 Analysis in capacity planning, 375–376 causal, 301–302 crash reports, 129 in monitoring, 345, 353–354 Ancillary resources in capacity planning, 372 Andreessen, Marc, 181 Anomaly detection, 354 “Antifragile Organization” article, 315, 320 Antifragile systems, 308–310 Apache systems Hadoop, 132, 467 Mesos, 34 web server forking, 114 Zookeeper, 231, 363 API (Application Programming Interface) defined, 10 logs, 340 Applicability in dot-bomb era, 463–464 Application architectures, 69 cloud-scale service, 80–85 exercises, 93 four-tier web service, 77–80 message bus, 85–90 reverse proxy service, 80 service-oriented, 90–92 single-machine web servers, 70–71 summary, 92–93 three-tier web service, 71–77 Application debug logs, 340 Application logs, 340 Application Programming Interface (API) defined, 10 logs, 340 Application servers in four-tier web service, 79 Approvals code, 47–48 deployment phase, 214, 216–217 design documents, 277, 281 service launches, 159 Arbitrary groups, segmentation by, 104 ARC (Adaptive Replacement Cache) algorithm, 107 Architecture factors in service launches, 157 Archives design documents, 279–280 email, 277 Art of Scalability, Scalable Web Architecture, Processes, and Organizations for the Modern Enterprise, 100 Artifacts artifact-scripted database changes, 185 defined, 196 Assessments, 421–422 Capacity Planning, 431–432 Change Management, 433–434 Disaster Preparedness, 448–450 Emergency Response, 426–428 levels, 405–407 methodology, 403–407 Monitoring and Metrics, 428–430 New Product Introduction and Removal, 435–436 operational excellence, 405–407 organizational, 411–412 Performance and Efficiency, 439–441 questions, 407 Regular Tasks, 423–425 Service Delivery: The Build Phase, 442–443 Service Delivery: The Deployment Phase, 444–445 Service Deployment and Decommissioning, 437–438 services, 407–410 Toil Reduction, 446–447 Asynchronous design, 29 Atlassian Bamboo tool, 205 Atomicity ACID term, 24 release, 240–241 Attack surface area, 79 Auditing operations design, 42–43 Augmentation files, 41–42 Authentication, authorization, and accounting (AAA), 222 Authentication in deployment phase, 222 Authors in design documents, 277, 282 Auto manufacturing automation example, 251 Automation, 243–244 approaches, 244–245 baking, 219 benefits, 154 code amount, 269–270 code reviews, 268–269 compensatory principle, 246–247 complementarity principle, 247–248 continuous delivery, 190 crash data collection and analysis, 129 creating, 255–258 DevOps, 182, 185–186 exercises, 272–273 goals, 252–254 hidden costs, 250 infrastructure strategies, 217–220 issue tracking systems, 263–265 language tools, 258–262 left-over principle, 245–246 lessons learned, 249–250 multitenant systems, 270–271 prioritizing, 257–258 repair life cycle, 254–255 software engineering tools and techniques, 262–270 software packaging, 266 software restarts and escalation, 128–129 steps, 258 style guides, 266–267, 270 summary, 271–272 system administration, 248–249 tasks, 153–155 test-driven development, 267–268 toil reduction, 257 vs. tool building, 250–252 version control systems, 265–266 Availability CAP Principle, 21–22 monitoring, 336 Availability and partition tolerance (AP), 24 Availability requirements cloud computing era, 469 dot-bomb era, 460 first web era, 455 pre-web era, 452–453 second web era, 465 Averages in monitoring, 358 AWS (Amazon Web Services), 59 Backend replicas, load balancers with, 12–13 Backends multiple, 14–15 server stability, 336 Background in design documents, 277–278 Background processes for containers, 61 Backups in design for operations, 36 Baked images for OS installation, 219–220 Banned query lists, 130 Bare metal clouds, 68 Barroso, L. A. canary requests, 131 cost comparisons, 464 disk drive failures, 133, 338 BASE (Basically Available Soft-state services) databases, 24 Baseboard Management Controller (BMC), 218 Basecamp application, 55 bash (Bourne Again Shell), 259 Batch size in DevOps, 178–179 Bathtub failure curve, 133 Beck, K., 189 Behaviors in KPIs, 390–391 Behr, K., 172 Bellovin, S.


pages: 161 words: 39,526

Applied Artificial Intelligence: A Handbook for Business Leaders by Mariya Yao, Adelyn Zhou, Marlene Jia

Airbnb, Amazon Web Services, artificial general intelligence, autonomous vehicles, business intelligence, business process, call centre, chief data officer, computer vision, conceptual framework, en.wikipedia.org, future of work, industrial robot, Internet of things, iterative process, Jeff Bezos, job automation, Marc Andreessen, natural language processing, new economy, pattern recognition, performance metric, price discrimination, randomized controlled trial, recommendation engine, self-driving car, sentiment analysis, Silicon Valley, skunkworks, software is eating the world, source of truth, speech recognition, statistical model, strong AI, technological singularity

Many of the tech giants offer to “democratize AI” by releasing open-source development tools such as Keras, TensorFlow, CTNK, and PyTorch, or offer enterprise cloud solutions and proprietary Machine Learning as a Service (MLaaS) platforms. For companies that don’t have the infrastructure or the technical knowledge, these represent excellent solutions for quickly integrating AI capabilities into your company’s workflow . If you already store data and build applications on Amazon Web Services (AWS), Microsoft Azure, Google Cloud, or Apple’s iOS platform, using tightly integrated machine learning solutions like AWS Rekognition or Apple’s Core ML can simplify work for your own developers and may be the most economical business decision. However, the mere existence of a solution does not mean that it’s definitely right for your company. In the digital era, data is the hottest commodity.


pages: 302 words: 82,233

Beautiful security by Andy Oram, John Viega

Albert Einstein, Amazon Web Services, business intelligence, business process, call centre, cloud computing, corporate governance, credit crunch, crowdsourcing, defense in depth, Donald Davies, en.wikipedia.org, fault tolerance, Firefox, loose coupling, Marc Andreessen, market design, MITM: man-in-the-middle, Monroe Doctrine, new economy, Nicholas Carr, Nick Leeson, Norbert Wiener, optical character recognition, packet switching, peer-to-peer, performance metric, pirate software, Robert Bork, Search for Extraterrestrial Intelligence, security theater, SETI@home, Silicon Valley, Skype, software as a service, statistical model, Steven Levy, The Wisdom of Crowds, Upton Sinclair, web application, web of trust, zero day, Zimmermann PGP

Backing up to a central server is all well and good, but what happens if we get robbed and someone steals the PCs and the server? Enter the new world of web services and cloud computing. To mitigate the risk of catastrophic system loss, I wrote a simple plug-in (see the later section “Platforms of the Long-Tail Variety: Why the Future Will Be Different for Us All” on page 165) to the home server that makes use of the Amazon Web Services platform. At set intervals, the system copies the directories I chose onto the server and connects via web services to Amazon’s S3 (Simple Storage System) cloud infrastructure. The server sends a backup copy of the data I choose to the cloud. I make use of the WS-Security specification (and a few others) for web services, ensuring the data is encrypted and not tampered with in transport, and I make use of an X.509 digital certificate to ensure I am communicating with Amazon.

He served on the Roundtable on Scientific Communication and National Security, a collaborative project of the National Research Council and the Center for Strategic and International Studies. 268 CONTRIBUTORS INDEX Numbers 3-D Secure protocol account holder domain, 76 acquirer domain, 76 e-commerce security and, 76–78 evaluation of, 77 issuer domain, 76 transaction process, 76 802.11b standard, 51, 52 802.11i standard, 51 A ABA (American Bar Association), 203 Access Control Server (ACS), 77 accountability, 213, 214 ACS (Access Control Server), 77 ActionScript, 93 ad banners (see banner ads) Adams, Douglas, 158 Advanced Monitor System (AMS), 254, 256 advertising (see online advertising) adware (see spyware) Aegenis Group, 66 Agriculture, Department of, 196 AHS (Authentication History Server), 77 AI (artificial intelligence), 254, 257 AllowScriptAccess tag, 94 Amazon Web Services platform, 152 Amazon.com, 102 American Bar Association (ABA), 203 AMS (Advanced Monitor System), 254, 256 analyst confirmation traps, 12 Anderson, Chris, 165 Andreessen, Marc, 165, 166 Anna Carroll (barge), 206 anti-executables, 253 anti-spyware software evolution of, 251 initial implementation, 251 intrusive performance, 254 strict scrutiny, 252 anti-virus software diminished effectiveness, 249 functional fixation, 15 functionality, 232 historical review, 248–249 honeyclients and, 141 intrusive performance, 254 malware signature recognition, 251 need for new strategies, 248 strict scrutiny, 252 zero-day exploits and, 252 Apgar score, 37 Apgar, Virginia, 37 Apple Computer, 8 artificial intelligence (AI), 254, 257 Ascom-Tech AG, 117 Ashenfelter, Orley, 164 Aspect Security, 188 Atkins, Derek, 119 ATMs, early security flaws, 36 attacks (see malicious attacks) attribute certificates, 111 Attrition.org, 55 authentication 3-D Secure protocol, 77 auto-update and, 15 CV2 security code, 76 e-commerce security, 83, 84 federated programs, 210 NTLM, 6 password security, 7 PGP Global Directory and, 127 portability of, 85 security pitfall in, 71 SET protocol, 78 WEP support, 52 Authentication History Server (AHS), 77 authoritative keys, 123 authorization We’d like to hear your suggestions for improving our indexes.


pages: 484 words: 114,613

No Filter: The Inside Story of Instagram by Sarah Frier

Airbnb, Amazon Web Services, blockchain, Clayton Christensen, cloud computing, cryptocurrency, Donald Trump, Elon Musk, Frank Gehry, Jeff Bezos, Marc Andreessen, Mark Zuckerberg, Menlo Park, Minecraft, move fast and break things, move fast and break things, Network effects, new economy, Oculus Rift, Peter Thiel, ride hailing / ride sharing, side project, Silicon Valley, Silicon Valley startup, Snapchat, Steve Jobs, TaskRabbit, Tony Hsieh, Travis Kalanick, ubercab, Zipcar

Krieger nodded and smiled, not indulging Systrom’s fear, mostly because it was unproductive. With the unexpected deluge of users, they needed to think fast to keep Instagram online. Systrom called D’Angelo, the early Facebook chief technology officer and early investor in Instagram, for his advice. It was the first of several calls that day. Every hour, Instagram seemed to grow faster. D’Angelo eventually helped the company transition to renting server space from Amazon Web Services instead of buying their own. Within the first day, 25,000 people were using Instagram. Within the first week, it was 100,000, and Systrom had the surreal experience of seeing a stranger scrolling through the app on a San Francisco bus. He and Krieger started an Excel spreadsheet that would update live with each user added. Launch success is rarely a sign of an app’s longevity. People download new apps, get excited, and forget to open them again.

ABC News, 211 Above Category cycling, 185 abuse, abuse content, 41, 97, 261 Academy Awards, 152 Systrom at, 191–92, 204 Accenture, 260 Acton, Brian, 125, 256, 258 Adams, William (will.i.am), 128 Adidas, xix Adobe Lightroom, 240, 243 Adobe Photoshop, 21, 23, 244 advertising, 59, 176, 256 false, 244; see also fake news FB’s business of, 75, 77, 89, 91–92, 94, 96, 105, 118–19, 125, 149–50, 163, 217, 224, 277 IG’s business of, 104, 118–21, 124, 151, 155, 163–65, 174, 175–76, 184, 225, 241, 277 mobile, 74–75 television, 215 see also brand advertising advertising agencies, 89 FB’s relationship with, 120–21, 124 Ahrendts, Angela, 147 @aidanalexander, 171 AiGrow, 246 Airbnb, xvi, 45 Alba, Jessica, 130, 250 Alexander, Aidan, 171 algorithms: FB’s use of, 91, 103, 128, 162, 163, 208, 209, 210–12, 215, 221, 224, 259 IG’s early lack of, 34, 143 IG’s use of, 81, 170, 174, 197–98, 218, 229, 230–32, 233, 251, 271 IG users’ mistrust of, 197 YouTube’s use of, 233–34 @alittlepieceofinsane, 161 Allen, Nick, 117 Allen & Company, 49 Amanpour, Christiane, 127 Amaro photo filter, 23 Amazon, 22, 28, 139, 242 Communications Decency Act and, 41 Whole Foods acquired by, 64 Zappos acquired by, 105 Amazon Web Services, 26, 79–80 American Medical Association, 244 American Society of Plastic Surgeons, 244 analytics, 90, 102, 226 IG’s use of, 100, 178, 183, 226 IG users’ access to, 275–76 Anchor Psychology, 172 Anderson, Steve, 34 early IG investment of, 11, 15 on IG board, 37, 56, 63–64 Anderson Cooper 360, 142 Andreessen, Marc, 11 Andreessen Horowitz: early investment in IG by, 11, 15, 33 investment in PicPlz by, 33–34, 36, 77 Android, 19, 33, 110, 203 IG app for, 50, 51 angel investments, 16, 17, 24, 36 anonymity, user, 41, 80 on Formspring, 40 on IG, 41, 80, 163, 173, 218, 219, 260, 261 troubling content and, 40, 163, 218–19, 260; see also bullying antitrust laws, 75, 268 Antonow, Eric, 165 AOL, 116 Apple, 10, 21, 56, 65, 147, 167, 234 Apple app store, 26, 28, 38, 115 Apple IDs, 42 apps: filter, see filter apps, photo location-based, 15 see also mobile apps; specific apps Argentina, 12 Arthur D.


pages: 409 words: 112,055

The Fifth Domain: Defending Our Country, Our Companies, and Ourselves in the Age of Cyber Threats by Richard A. Clarke, Robert K. Knake

A Declaration of the Independence of Cyberspace, Affordable Care Act / Obamacare, Airbnb, Albert Einstein, Amazon Web Services, autonomous vehicles, barriers to entry, bitcoin, Black Swan, blockchain, borderless world, business cycle, business intelligence, call centre, Cass Sunstein, cloud computing, cognitive bias, commoditize, computer vision, corporate governance, cryptocurrency, data acquisition, DevOps, don't be evil, Donald Trump, Edward Snowden, Exxon Valdez, global village, immigration reform, Infrastructure as a Service, Internet of things, Jeff Bezos, Julian Assange, Kubernetes, Mark Zuckerberg, Metcalfe’s law, MITM: man-in-the-middle, move fast and break things, move fast and break things, Network effects, open borders, platform as a service, Ponzi scheme, ransomware, Richard Thaler, Sand Hill Road, Schrödinger's Cat, self-driving car, shareholder value, Silicon Valley, Silicon Valley startup, Skype, smart cities, Snapchat, software as a service, Steven Levy, Stuxnet, technoutopianism, Tim Cook: Apple, undersea cable, WikiLeaks, Y2K, zero day

Google is so confident in its security capabilities that, instead of arguing that it shouldn’t be expected to be able to stop government intelligence organizations, it is actively working to protect its customers from them and will notify individual Google account holders if they are being targeted by an APT actor. The danger with cloud computing is that it is concentrating risk in the hands of a few players that now have a near monopoly. Almost all SaaS providers start out building their services on top of Amazon Web Services or Microsoft Azure and many stay that way. Netflix, now in a heated rivalry with Amazon Prime for eyeballs in the streaming wars, uses Amazon, as do other giants of the internet age such as Airbnb. Dropbox, the online file storage company, until a few years ago was also an Amazon customer. What this concentration of risk means is that a problem at Amazon (or Microsoft or Google) could be a problem for everyone.

These might be compromised computers in homes and small businesses. They might have used a commercial or consumer-grade virtual private network or one of the many censorship circumvention tools designed to allow people behind national firewalls to roam the web freely. Or, if they were hip to the latest trends in the cyber-criminal underworld, they might have used a stolen credit-card number bought for fifty cents on the dark web to set up an Amazon Web Services account and purchased all the computing power they needed. Chances are, they used several of these techniques to create a string of hop points. What that means for the counteroffense team at Fort Meade (or, in some darker fantasies, at the utility) is that hitting the Iranians back just became pretty complicated. At the very least, the last hop point used to access the utility’s server was in the United States.


The Code: Silicon Valley and the Remaking of America by Margaret O'Mara

"side hustle", A Declaration of the Independence of Cyberspace, accounting loophole / creative accounting, affirmative action, Airbnb, AltaVista, Amazon Web Services, Apple II, Apple's 1984 Super Bowl advert, autonomous vehicles, back-to-the-land, barriers to entry, Ben Horowitz, Berlin Wall, Bob Noyce, Buckminster Fuller, Burning Man, business climate, Byte Shop, California gold rush, carried interest, clean water, cleantech, cloud computing, cognitive dissonance, commoditize, computer age, continuous integration, cuban missile crisis, Danny Hillis, DARPA: Urban Challenge, deindustrialization, different worldview, don't be evil, Donald Trump, Doomsday Clock, Douglas Engelbart, Dynabook, Edward Snowden, El Camino Real, Elon Musk, en.wikipedia.org, Erik Brynjolfsson, Frank Gehry, George Gilder, gig economy, Googley, Hacker Ethic, high net worth, Hush-A-Phone, immigration reform, income inequality, informal economy, information retrieval, invention of movable type, invisible hand, Isaac Newton, Jeff Bezos, Joan Didion, job automation, job-hopping, John Markoff, Julian Assange, Kitchen Debate, knowledge economy, knowledge worker, Lyft, Marc Andreessen, Mark Zuckerberg, market bubble, mass immigration, means of production, mega-rich, Menlo Park, Mikhail Gorbachev, millennium bug, Mitch Kapor, Mother of all demos, move fast and break things, move fast and break things, mutually assured destruction, new economy, Norbert Wiener, old-boy network, pattern recognition, Paul Graham, Paul Terrell, paypal mafia, Peter Thiel, pets.com, pirate software, popular electronics, pre–internet, Ralph Nader, RAND corporation, Richard Florida, ride hailing / ride sharing, risk tolerance, Robert Metcalfe, Ronald Reagan, Sand Hill Road, Second Machine Age, self-driving car, shareholder value, side project, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, skunkworks, Snapchat, social graph, software is eating the world, speech recognition, Steve Ballmer, Steve Jobs, Steve Wozniak, Steven Levy, Stewart Brand, supercomputer in your pocket, technoutopianism, Ted Nelson, the market place, the new new thing, There's no reason for any individual to have a computer in his home - Ken Olsen, Thomas L Friedman, Tim Cook: Apple, transcontinental railway, Uber and Lyft, uber lyft, Unsafe at Any Speed, upwardly mobile, Vannevar Bush, War on Poverty, We wanted flying cars, instead we got 140 characters, Whole Earth Catalog, WikiLeaks, William Shockley: the traitorous eight, Y Combinator, Y2K

ELLEN ULLMAN, Life in Code, 19983 CONTENTS Also by Margaret O’Mara Title Page Copyright Dedication Epigraph List of Abbreviations Introduction: The American Revolution ACT ONE: START UP Arrivals Chapter 1: Endless Frontier Chapter 2: Golden State Chapter 3: Shoot the Moon Chapter 4: Networked Chapter 5: The Money Men Arrivals Chapter 6: Boom and Bust ACT TWO: PRODUCT LAUNCH Arrivals Chapter 7: The Olympics of Capitalism Chapter 8: Power to the People Chapter 9: The Personal Machine Chapter 10: Homebrewed Chapter 11: Unforgettable Chapter 12: Risky Business ACT THREE: GO PUBLIC Arrivals Chapter 13: Storytellers Chapter 14: California Dreaming Chapter 15: Made in Japan Chapter 16: Big Brother Chapter 17: War Games Chapter 18: Built on Sand ACT FOUR: CHANGE THE WORLD Arrivals Chapter 19: Information Means Empowerment Chapter 20: Suits in the Valley Chapter 21: Magna Carta Chapter 22: Don’t Be Evil Arrivals Chapter 23: The Internet Is You Chapter 24: Software Eats the World Chapter 25: Masters of the Universe Departure: Into the Driverless Car Photographs Acknowledgments Note on Sources Notes Image Credits Index About the Author LIST OF ABBREVIATIONS ACM: Association for Computing Machinery AEA: American Electronics Association AI: Artificial intelligence AMD: Advanced Micro Devices ARD: American Research and Development ARM: Advanced reduced-instruction-set microprocessor ARPA: Advanced Research Projects Agency, Department of Defense, renamed DARPA AWS: Amazon Web Services BBS: Bulletin Board Services CDA: Communications Decency Act of 1996 CPSR: Computer Professionals for Social Responsibility CPU: Central processing unit EDS: Electronic Data Systems EFF: Electronic Frontier Foundation EIT: Enterprise Integration Technologies ENIAC: Electronic Numerical Integrator and Computer ERISA: Employee Retirement Income Security Act of 1974 FASB: Financial Accounting Standards Board FCC: Federal Communications Commission FTC: Federal Trade Commission GUI: Graphical user interface HTML: Hypertext markup language IC: Integrated circuit IPO: Initial public offering MIS: Management information systems MITI: Ministry of International Trade and Industry (of Japan) NACA: National Advisory Committee for Aeronautics, later superseded by NASA NASA: National Aeronautics and Space Administration NASD: National Association of Securities Dealers NDEA: National Defense Education Act NII: National Information Infrastructure NSF: National Science Foundation NVCA: National Venture Capital Association OS: Operating system OSRD: U.S.

The company had upended the publishing industry and was pushing into new realms, delivering value and convenience for its customers while it chased brick-and-mortar stores out of business. A big part of its growth came from turning itself into a platform for third-party buying and selling, giving businesses small and large an opportunity to reach Amazon’s enormous audience. Now Amazon was branching out further into large-scale software platforms. The biggest of them all was Amazon Web Services, or AWS. When he talked about AWS, Bezos sounded a lot like Steve Jobs talking about the Apple II. “The most radical and transformative of inventions are often those that empower others to unleash their creativity—to pursue their dreams,” he told shareholders in 2011. Amazon had launched the service without much hoopla in 2006, targeting a new set of customers: software developers in search of storage and sophisticated computer power.

Advanced Micro Devices (AMD), 3, 101, 207, 208, 239, 244 Adweek, 315 AdWords and AdSense, 365 Air Force, 37, 38, 57 Albrecht, Bob, 118, 128, 134, 140 Alger, Horatio, 4, 170 Allaire, Paul, 296 Allen, Paul, 154, 155, 227–29, 351 Allison, Dennis, 140 Alsop, Stewart, 349 Altair, 135–36, 138–40, 142, 144–46, 154, 155, 227 AltaVista, 362, 365 Alto, 130, 131, 234, 243 Amazon, 1, 2, 221, 313–15, 317, 324, 354, 359, 366, 380–83, 389, 391, 392, 394, 396, 408, 410 Amazon Web Services (AWS), 381–83, 390–91 American Bankers’ Association, 160 American Challenge, The (Servan-Schreiber), 88, 89 American Electronics Association (AEA), 168, 171, 197, 350 American Research and Development (ARD), 70, 71 America Online (AOL), 306, 316, 317, 346, 358, 368–71 Ampex, 38, 74, 207, 249, 262 Anderson, Fred, 71, 72 Anderson, Harlan, 54 Andreessen, Marc, 2, 305, 306, 309, 316, 318, 338, 341–43, 348, 370, 390, 392–93, 399 Andrews, Paul, 155 Android, 378 Anokwa, Yaw, 408–10 Apollo program, 51, 67, 86, 396 Apple, 1–3, 42, 146–52, 154, 157, 178–84, 186–90, 191, 192, 194, 199–201, 207, 215, 217–21, 223, 227, 229, 231–37, 240–44, 247, 249, 257, 264–67, 270, 271, 273, 277, 280, 284, 289, 295, 304, 305, 322, 328, 336, 355, 356, 364, 366, 372, 375–79, 388, 391, 392, 394, 395, 401, 405, 410 Apple Bill, 218–19, 224 Armstrong, Neil, 65 Army, U.S., 24 ARPA (Advanced Research Projects Agency; renamed DARPA), 45, 57–58, 64, 133, 225, 226, 227, 246, 248, 287, 288, 312, 352 ARPANET, 64–66, 67, 130, 257, 258, 287, 304 Arrillaga, John, 80, 264 Asimov, Isaac, 257 Association for Computing Machinery, 125 AT&T, 61–64, 118, 129, 167, 193, 238, 305, 328 Atari, 106–8, 147, 149, 154, 185, 201 Atari Democrats, 193–94, 216, 217, 221, 222, 224, 290, 325 @Home, 306 Atlantic, 20, 58 Augmentation Research Center, 91 Auletta, Ken, 364 Ayres, Judith, 263 Azure, 383 Babbitt, Bruce, 193 Badham, John, 246–47 Ballmer, Steve, 228, 273, 340–42, 350, 366, 377 Bancroft, Pete, 161–63 Banham, Reyner, 198 Bank of America, 74, 88 Baran, Paul, 124 Barksdale, Jim, 305, 344 Barlow, John Perry, 258, 286–87, 291, 301, 327 Barram, Dave, 294–95, 297, 298 Bayh-Dole Act, 180 Bechtolsheim, Andy, 275, 277, 354 Bell Laboratories, 40, 364 Bentsen, Lloyd, 170–71 Berezin, Evelyn, 124 Berlin, Leslie, 105 Berman, Jerry, 301 Berners-Lee, Tim, 287–90, 305 Bezos, Jeff, 311–15, 354, 355, 380–82, 391, 402 Bidzos, Jim, 310, 311 Billboard, 357 Billionaire Boys Club, 285 Black, Shirley Temple, 79 Blackberry, 376 Blacks at Microsoft, 320, 322 Blodget, Henry, 359 Bloom, Allan, 253 Bloomberg, Michael, 402–3 Bloomfield, Mark, 168, 169, 222 Blue Origin, 402 Blumenthal, Michael, 170 Boeing, 29, 89–90, 232, 271, 314, 384 Boggs, David, 129–30 Boxer, Barbara, 331 Brand, Stewart, 118, 128, 130, 142, 247, 258 Bricklin, Dan, 188 Bridges, Harry, 48 Brin, Sergey, 351–55, 362–65, 373, 375, 404 Brown, Dean, 117, 125, 127, 128, 134–35 Brown, Jerry, 142–43, 156, 194, 212–14, 216, 219, 224, 225, 292, 293 Brown, Pat, 81, 142 Brown, Ron, 299 Bucy, Fred, 211 Bunker, George, 63 Bunker Ramo Corporation, 63, 64, 238 Bunnell, David, 139–40 Burning Man, 363, 369 Bush, George H.


pages: 138 words: 40,787

The Silent Intelligence: The Internet of Things by Daniel Kellmereit, Daniel Obodovski

Airbnb, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, autonomous vehicles, barriers to entry, business intelligence, call centre, Clayton Christensen, cloud computing, commoditize, connected car, crowdsourcing, data acquisition, en.wikipedia.org, Erik Brynjolfsson, first square of the chessboard, first square of the chessboard / second half of the chessboard, Freestyle chess, Google X / Alphabet X, Internet of things, lifelogging, Metcalfe’s law, Network effects, Paul Graham, Ray Kurzweil, RFID, Robert Metcalfe, self-driving car, Silicon Valley, smart cities, smart grid, software as a service, Steve Jobs, web application, Y Combinator, yield management

Most companies providing asset tracking and monitoring services do it in a SaaS way, again bypassing the IT departments and providing much more cost-effective and accurate solutions that way. But speaking of security of connectivity and information, Sanjay Sarma believes cloud services are not any less secure than the ones provided by corporate IT: I think that outsourcing innovation actually ensures security, because you have a few professionals that have best practices. I’d rather trust a thousand people at a company like Amazon Web Services than the four guys in an IT division who have been working there for twenty years. The cloud companies will have the latest enterprise software on their machines, including the latest security updates. That is particularly true if you are doing certain things with sensors that would generate gazillions of terabytes, so much data to be interpreted, analyzed, etc. Indeed, internal IT departments might not have capacity and capability to deal with these volumes of data.


pages: 179 words: 42,006

Startup Weekend: How to Take a Company From Concept to Creation in 54 Hours by Marc Nager, Clint Nelsen, Franck Nouyrigat

Amazon Web Services, barriers to entry, business climate, invention of the steam engine, James Watt: steam engine, Mark Zuckerberg, minimum viable product, pattern recognition, Silicon Valley, transaction costs, web application, Y Combinator

The following list is by no means complete: Andy Sack, Bill Warner, Brad Feld, Dave McClure, Denis Browne, Eric Ries, Jessica Livingston, John Lewis, Jonathan Ortmans, Kathleen Kennedy, Mark Suster, Robert Scoble, and Yosi Vardi. We&apos;d be remiss if we didn&apos;t acknowledge the huge impact our global sponsors have had not only on the organization but on the thousands of Startup Weekend alumni, too. Thank you to Amazon Web Services (Rodica Buzescu), oDesk, O&apos;Reilly, Microsoft BizSpark (particularly Juliano Tubino, Julien Codiniou, Ludo Ulrich, and the rest of the global team), Sun Microsystems (particularly Jeremiah Shackelford), TokBox, and Twilio. Another round of thanks is also in order for the regional and local sponsors who help us bring our events to cities around the world. A huge thank you goes out to the Startup Weekend Core Team: Keith Armstrong, Jennifer Cabala, Anca Foster, Ashley Hodgson, Maris McEdward, Joey Pomerenke, Tawnee Rebhuhn, Shane Reiser, and Adam Stelle for their belief in and commitment to our vision.


pages: 179 words: 43,441

The Fourth Industrial Revolution by Klaus Schwab

3D printing, additive manufacturing, Airbnb, Amazon Mechanical Turk, Amazon Web Services, augmented reality, autonomous vehicles, barriers to entry, Baxter: Rethink Robotics, bitcoin, blockchain, Buckminster Fuller, call centre, clean water, collaborative consumption, commoditize, conceptual framework, continuous integration, crowdsourcing, digital twin, disintermediation, disruptive innovation, distributed ledger, Edward Snowden, Elon Musk, epigenetics, Erik Brynjolfsson, future of work, global value chain, Google Glasses, income inequality, Internet Archive, Internet of things, invention of the steam engine, job automation, job satisfaction, John Maynard Keynes: Economic Possibilities for our Grandchildren, John Maynard Keynes: technological unemployment, life extension, Lyft, mass immigration, megacity, meta analysis, meta-analysis, more computing power than Apollo, mutually assured destruction, Narrative Science, Network effects, Nicholas Carr, personalized medicine, precariat, precision agriculture, Productivity paradox, race to the bottom, randomized controlled trial, reshoring, RFID, rising living standards, Sam Altman, Second Machine Age, secular stagnation, self-driving car, sharing economy, Silicon Valley, smart cities, smart contracts, software as a service, Stephen Hawking, Steve Jobs, Steven Levy, Stuxnet, supercomputer in your pocket, TaskRabbit, The Future of Employment, The Spirit Level, total factor productivity, transaction costs, Uber and Lyft, uber lyft, Watson beat the top human players on Jeopardy!, WikiLeaks, winner-take-all economy, women in the workforce, working-age population, Y Combinator, Zipcar

One reason for it is that the storage price (Figure IV) has dropped exponentially (by a factor of approximately ten, every five years). Figure IV: Hard Drive Cost per Gigabyte (1980-2009) Source: “a history of storage costs”, mkomo.com, 8 September 200988 An estimated 90% of the world’s data has been created in the past two years, and the amount of information created by businesses is doubling every 1.2 years.89 Storage has already become a commodity, with companies like Amazon Web Services and Dropbox leading this trend. The world is heading towards a full commoditization of storage, through free and unlimited access for users. The best-case scenario of revenue for companies could potentially be advertising or telemetry. Positive impacts – Legal systems – History scholarship/academia – Efficiency in business operations – Extension of personal memory limitations Negative impact – Privacy surveillance Unknown, or cuts both ways – Eternal memory (nothing deleted) – Increased content creation, sharing and consumption The shift in action Numerous companies already offer free storage in the cloud, ranging from 2 GB to 50 GB.


pages: 525 words: 116,295

The New Digital Age: Transforming Nations, Businesses, and Our Lives by Eric Schmidt, Jared Cohen

access to a mobile phone, additive manufacturing, airport security, Amazon Mechanical Turk, Amazon Web Services, anti-communist, augmented reality, Ayatollah Khomeini, barriers to entry, bitcoin, borderless world, call centre, Chelsea Manning, citizen journalism, clean water, cloud computing, crowdsourcing, data acquisition, Dean Kamen, drone strike, Elon Musk, failed state, fear of failure, Filter Bubble, Google Earth, Google Glasses, hive mind, income inequality, information trail, invention of the printing press, job automation, John Markoff, Julian Assange, Khan Academy, Kickstarter, knowledge economy, Law of Accelerating Returns, market fundamentalism, means of production, MITM: man-in-the-middle, mobile money, mutually assured destruction, Naomi Klein, Nelson Mandela, offshore financial centre, Parag Khanna, peer-to-peer, peer-to-peer lending, personalized medicine, Peter Singer: altruism, Ray Kurzweil, RFID, Robert Bork, self-driving car, sentiment analysis, Silicon Valley, Skype, Snapchat, social graph, speech recognition, Steve Jobs, Steven Pinker, Stewart Brand, Stuxnet, The Wisdom of Crowds, upwardly mobile, Whole Earth Catalog, WikiLeaks, young professional, zero day

Building platforms that we merely hope alienated youth will like and use is the equivalent of dropping propaganda flyers from an airplane. Outsiders don’t have to develop the content; they just need to create the space. Wire up the city, give people basic tools and they’ll do most of the work themselves. A number of technology companies have developed start-up kits for people to build applications on top of their platforms; Amazon Web Services and Google App Engine are two examples, and there will be many others. Creating space for others to build the businesses, games, platforms and organizations they envision is a brilliant corporate maneuver, because it ensures that a company’s products are used (boosting brand loyalty, too) while the users actually build and operate what they want. Somalis will build apps that are effective antiradicalization tools to reach other Somalis; Pakistanis will do the same for other Pakistanis.

INDEX Aadhaar Abbottabad, Pakistan, 2.1, 5.1 Abkhaz nationalists Abuja, Nigeria Academi, LLC accountability, 2.1, 4.1, 6.1, 7.1 activist groups additive manufacturing Advanced Research Projects Agency (ARPA), n Afghanistan, 1.1, 4.1, 5.1, 5.2, 5.3, 6.1, 6.2, 7.1 reconstruction of, 7.1, 7.2, 7.3 Africa, 3.1, 4.1, 4.2 African Americans African National Congress (ANC) African Sahel African Union Age of Spiritual Machines, The: When Computers Exceed Human Intelligence (Kurzweil), con.1 Agha-Soltan, Neda Agie, Mullah Akbar Agreement on Trade-Related Aspects of Intellectual Property Rights (1994) Ahmadinejad, Mahmoud al-Aqsa Martyrs Brigades al-Assad, Bashar Alcatel-Lucent AlertNet Algeria, 3.1, 4.1 alienation Al Jazeera al-Qaeda, 5.1, 5.2, 5.3, 5.4, 5.5, con.1 al-Shabaab, 2.1, 5.1, 7.1, 7.2 Amazon, itr.1, 1.1, 1.2 data safeguarded by Amazon Web Services American Sentinel drone Android anonymity, 2.1, 3.1, 4.1 Anonymous, 5.1, 5.2 Anti-Ballistic Missile Treaty antiradicalization antiterrorism units, 5.1, 5.2, 5.3, 5.4 Apple, itr.1, 5.1 data safeguarded by apps, 2.1, 5.1 Arab Spring, itr.1, 4.1, 4.2, 4.3, 4.4, 4.5 AR.Drone quadricopter Argentina Armenia arms-for-minerals trade arrests artificial intelligence (AI), itr.1, 1.1 artificial pacemakers Asia Asia-Pacific Economic Cooperation (APEC) Assange, Julian, 2.1, 2.2, 2.3, 2.4, 5.1 Astroturfing Atatürk, Mustafa Kemal, 3.1, 3.2 Athar, Sohaib, n, 269 ATMs augmented reality (AR), itr.1, 2.1 autocracies, 2.1, 3.1, 3.2 data revolution in dissent in information shared by online discussions in Ayalon, Danny Baghdad Baghdad Museum Bahrain Baidu.com, n Bamiyan Buddhas Bangladesh bank loans Basque separatists Batbold, Sukhbaatar battery life Bechtel Belarus Belgium Ben Ali, Zine el-Abidine, 4.1, 4.2 Berezovsky, Boris Better Angels of Our Nature, The (Pinker), 6.1 big data challenge Bill of Guarantees bin Laden, Osama, 2.1, 5.1, 5.2, 6.1, nts.1 biometric information, 2.1, 2.2, 6.1, 6.2, 6.3 Bitcoin, 2.1, nts.1 BlackBerry Messenger (BBM), 2.1, 2.2, 4.1, 5.1 Black Hat Blackwater Blockbuster, n Bloomberg News Bluetooth, 2.1, 2.2, 6.1 body scan body temperatures Boko Haram Bosnia brand Brand, Stewart, n Brazil, 5.1, 5.2, 5.3 Bush, George H.


pages: 382 words: 120,064

Bank 3.0: Why Banking Is No Longer Somewhere You Go but Something You Do by Brett King

3D printing, additive manufacturing, Airbus A320, Albert Einstein, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, asset-backed security, augmented reality, barriers to entry, bitcoin, bounce rate, business intelligence, business process, business process outsourcing, call centre, capital controls, citizen journalism, Clayton Christensen, cloud computing, credit crunch, crowdsourcing, disintermediation, en.wikipedia.org, fixed income, George Gilder, Google Glasses, high net worth, I think there is a world market for maybe five computers, Infrastructure as a Service, invention of the printing press, Jeff Bezos, jimmy wales, Kickstarter, London Interbank Offered Rate, M-Pesa, Mark Zuckerberg, mass affluent, Metcalfe’s law, microcredit, mobile money, more computing power than Apollo, Northern Rock, Occupy movement, optical character recognition, peer-to-peer, performance metric, Pingit, platform as a service, QR code, QWERTY keyboard, Ray Kurzweil, recommendation engine, RFID, risk tolerance, Robert Metcalfe, self-driving car, Skype, speech recognition, stem cell, telepresence, Tim Cook: Apple, transaction costs, underbanked, US Airways Flight 1549, web application

—Carmen Herranz, director of innovation at BBVA6 However, much earlier than the BBVA move, some of the more progressive institutions were already floating the concept of moving core banking systems into the cloud for the same reasons—improved productivity, decision making, portability and speed. In May 2010, Michael Harte, CIO of Commonwealth Bank in Australia, announced CBA’s intention to set up a cloud-based operation with Amazon Web Services. Harte explained the rationale behind this move as looking to reduce the cost of purchasing IT and related infrastructure by paying for services on demand as CBA grew, especially as reliance on more digital integration and real-time engagement became essential to CBA’s customer experience. In December 2011, Deutsche Bank went live with its first phase of cloud deployment, namely its IaaS (Infrastructure as a Service) development platform.

Deutsche Bank, mentioned earlier, has also developed new modular data centre designs and elastic computing platforms, underpinned by its core identity management platforms, including a Microsoft Active Directory system and an SAP-linked Global LDAP directory. Bank of America, in its efforts to build up private cloud capability, ran just south of 90,000 physical servers and 40 per cent of them were running in the cloud. BofA has set the target to create scaling and capability similar to that of Amazon Web Services environment too, according to Brad Spiers, Head of Compute Innovation at BofA. Interesting to note is that BofA is heavily investing in graphics processing capability, solid-state storage and in-memory databases, with large, fast processing and decision-making capability as the objective. NAB (previously National Australia Bank) of Australia has also committed extensively to private cloud infrastructure as it has moved to a platform-as-a-service concept as part of its programmes built around what it calls NextGen.


pages: 468 words: 124,573

How to Build a Billion Dollar App: Discover the Secrets of the Most Successful Entrepreneurs of Our Time by George Berkowski

Airbnb, Amazon Web Services, barriers to entry, Black Swan, business intelligence, call centre, crowdsourcing, disruptive innovation, en.wikipedia.org, game design, Google Glasses, Google Hangouts, Google X / Alphabet X, iterative process, Jeff Bezos, Jony Ive, Kickstarter, knowledge worker, Lean Startup, loose coupling, Marc Andreessen, Mark Zuckerberg, minimum viable product, MITM: man-in-the-middle, move fast and break things, move fast and break things, Network effects, Oculus Rift, Paul Graham, QR code, Ruby on Rails, self-driving car, Silicon Valley, Silicon Valley startup, Skype, Snapchat, social graph, software as a service, software is eating the world, Steve Jobs, Steven Levy, Travis Kalanick, ubercab, Y Combinator

At Amazon, Bezos mandated that every internal product or feature should have an API. This means that it can be more easily shared (and used) by internal product and development teams. But it also means that the option exists to allow external developers to use it as well. The added bonus is that APIs lay the groundwork for the productisation of internal tools. That simple declaration created an IT, as well as cultural, architecture that catalysed the growth of Amazon Web Services. Within a few short years of its launch in 2006, the service was already a billion-dollar business.2 In short, small teams can run fast and innovate because of their size and the fact that they’re not reliant on the technology from other teams. Move Fast and Break Things Facebook created a culture of agility, promoting a philosophy to ‘move fast and break things’.3 Mark Zuckerberg explained the company’s ‘hacker way’ in a letter to investors:4 Hackers try to build the best services over the long term by quickly releasing and learning from smaller iterations rather than trying to get everything right all at once … We have the words ‘Done is better than perfect’ painted on our walls to remind ourselves to always keep shipping.

Index Note: page numbers in bold refer to illustrations, page numbers in italics refer to information contained in tables. 99designs.com 111 500 Startups accelerator 136, 160 Accel Partners 3, 158, 261, 304, 321, 336, 383 accelerators 136, 159–60, 160 accountants 164, 316 accounting software 164 acquisition (of users) costs 148–9, 184, 236–7, 275–9, 282 and Facebook 271, 272, 273–4 for five hundred-million-dollar apps 327, 341–3 for hundred-million-dollar apps 252, 259, 266, 267–74, 275–84, 295–307 and incentive-based networks 270–1 international 295–307 for million dollar apps 136–7, 139, 140–51, 148–9, 153 and mobile social media channels 271–3, 272 and mobile user-acquisition channels 269–70 strategy 222–31 for ten-million-dollar apps 211–12, 213, 222–31, 236–7, 248–9 and traditional channels 268–9 and ‘viral’ growth 225, 278, 279–84 zero-user-acquisition cost 278 acquisitions 414–25 buying sustained growth 417–18 by non-tech corporations 418–20 initial public offerings 420–2 Waze 415–16 activation (user) 136, 137, 139, 153–4, 211–12, 213 Acton, Brian 54, 394 addiction, smartphone 30–1 Adler, Micah 269 administrators 409 AdMob 414–15 advertising 43 business model 67, 89–90 costs 140 and Facebook 271, 272, 273–4 mobile 148–9, 268–70, 272–3, 272 mobile social media 272–3, 272 mobile user-acquisition channels 269–70 outdoor 264 shunning of 42, 54–6 video ads 273 aesthetics 131 after product–market fit (APMF) 180 agencies 195–7, 264, 343 ‘agile coaches’ see scrum masters agile software development 192–3, 299, 315, 357, 377 Ahonen, Tomi 45 ‘aiming high’ 40–1 Airbnb 160, 301 alarm features 48 Albion 111 alerts 293 Alexa.com 146 Alibaba 227 ‘ALT tags’ 147 Amazon 7, 29, 131, 164, 227, 276, 366, 374–5, 401, 406 Amazon Web Services 374 American Express 347 Amobee 149 analytics 134–5, 149, 199, 205, 210, 212, 217–21, 294 and cohort analysis 287–8 Flurry 135, 149, 220 function 217–18 Google Analytics 135, 219–20, 345 limitations 284 Localytics 135, 221 and marketing 263 mistakes involving 218–19 Mixpanel.com tool 135, 217–18, 220–1, 287, 290–1, 345 Andreessen, Marc 180, 418–19 Andreessen Horowitz 72, 80, 180, 321, 383, 385, 418–19 Android (mobile operating system) 6, 23–4, 38, 415 advertising 274 audience size 119 beta testing 202 building apps for 116–22 and international apps 296 in Japan 306 scaling development and engineering 357–8 time spent on 26 and WhatsApp 55 Angel Capital Association 162 angel investors 154, 155–6, 323 AngelList 99, 131, 155, 159, 233 Angry Birds (game) 6, 42, 47, 57–8, 87, 89, 97 and application programming interface 36 delivering delight 207 design 131 funding 321 game in game 348–9 international growth 297–9 platform 117, 118 product extension 356 virality 282 annual offsites 379 annual revenue per user (ARPU) 215, 219, 232, 236 anonymity 43, 56–7 anti-poaching clauses 247 antidilution rights 245 API see application programming interface app descriptions 143 app development billion-dollar app 8, 389–425 CEO advice 406–13 getting acquired 414–25 people 395–405 process 390–1 five-hundred-million-dollar app 325–87 funding 328, 383–7 hiring staff 334–6, 337–40 killer product expansion 350–63 process 326–8 scaling 326, 330–6, 331–2 scaling marketing 341–9 scaling people 364–72, 377–9 scaling process 373–82 scaling product development 357–63 hundred-million-dollar app 251–324 international growth 295–307 process 252–4 product-market fit 255–6 retention of users 286–94 revenue engines 257–66, 275–85 user acquisition 267–74 million-dollar app 81–171 app Version 0.1 123–35 coding 133–4 design 129–33 feedback 127, 134–5 funding 152–60, 161–71, 176, 235–49 identity of the business 106–14 lean companies 115–22 metrics 136–9, 139 process 82–4 startup process 85–105 testing 126–8 user acquisition 140–51 ten-million-dollar app 173–249 growth engine 222–31, 235–49 metrics 211–21 new and improved Version 1.0 198–210 process 174–6 product–market fit 180–97 revenue engine 232–4 venture capital 235–49 app stores 22, 27–8, 33–4 see also Apple App Store; Google Play app-store optimisation (ASO) 142, 225 AppAnnie 205 Apple 19, 20, 31–2, 393 application programming interface 35–6 designers 129 Facetime app 46 iWatch 38–9 profit per employee 402–3 revenue per employee 401 visual voicemail 50 Worldwide Developers Conference (WWDC) 313 see also iPad; iPhone Apple App Store 22, 27, 32–3, 75, 88, 89, 117, 226 finding apps in 140, 141, 142–5 international apps 297–9 making submissions to 152–3 and profit per employee 403 ratings plus comments 204–5 Apple Enterprise Distribution 201–2 application programming interface (API) 35–6, 185, 360, 374 ARPU see annual revenue per user articles of incorporation 169 ASO see app-store optimisation Atari 20 Atomico 3, 261, 321, 383 attribution 227–31 for referrals 230–1 average transaction value (ATV) 214–15, 219, 232, 236, 387 Avis 95 backlinking to yourself 146 ‘bad leavers’ 247 Balsamiq.com 128 Banana Republic 352 bank accounts 164 banking 156–7 Bardin, Noam 43 Barr, Tom 338 Barra, Hugo 120, 306 Baseline Ventures 72 Baudu 226 beauty 131 BeeJiveIM 33 before product–market fit (BPMF) 180 ‘below the fold’ 143 Beluga Linguistics 297 Benchmark 75 benefits 398–400 beta testing 201–4 Betfair 358 Bezos, Jeff 366, 374 Bible apps 45 billion 9–10 Billion-Dollar Club 5 billionaires 9 Bing 226 ‘black-swan’ events 54 BlackBerry 23 Blank, Steve 257 Blogger 41 blood sugar monitoring devices 38 board seats 242, 243–4 board-member election consent 169 Bolt Peters 363 Booking.com 320 Bootstrap 145 Botha, Roelof 76, 77, 80 Box 7, 90, 276, 396–7, 411 brains 10 brainstorming 108 branding 111–13, 143, 263–4 Braun 129 Bregman, Jay xiii, 14–16, 95, 124, 209, 303 bridge loans 323 Brin, Sergey 366 Bring Your Own Infrastructure (BYOI) 17–18 Brougher, Francoise 340 Brown, Donald 44 Brown, Reggie 104–5 Bubble Witch 421 Buffet, Warren 4 build-measure-learn cycle 116 Burbn.com 72–4, 80 business advisors/coaches 103 business analysts 343 business culture 395–8 business goal setting 310–11 business models 67, 83, 87, 88–91, 175, 253, 259, 327, 351–2, 391, 400, 423–4 business success, engines of 183–4, 423–4 Business Wire 150 CAC see Customer Acquisition Cost Cagan, Marty 314 calendars 49 calorie measurement sensors 38 Cambridge Computer Scientists 160 camera feature 48 Camera+ app 48 Candy Crush Saga 6, 47, 87, 89, 131, 278–81, 318, 349, 421–2 card-readers 41–2 cash flow 164 CEOs see Chief Executive Officers CFOs see Chief Financial Officers channels incentive-based networks 270–1 mobile social-media 271–3, 272 mobile user-acquisition 269–70 source attribution 227–31 testing 224–7 traditional 268–9 viral 280–2 charging phones 49–50 Chartboost 149 chauffer hire see Uber app check-ins, location-based 72, 74 Chief Executive Officers (CEOs) 309, 380 advice from 406–13 and the long haul 68 and product centricity 185–6 role 337 Chief Financial Officers (CFOs) 316 Chief Operations Officers (COOs) 309, 326, 337–40, 380 Chief Technology Officers (CTOs) 186–7, 195 Chillingo 298 China 24–5, 146, 226, 306–7 Cisco 402 Clash of Clans (game) 6, 28, 36, 47, 87, 89, 97, 118, 227, 348–9, 398 Clements, Dave 120 Climate Corporation 412, 419 clock features 47 cloud-based software 67, 90 Clover 419 coding 133–4 cofounders 85, 91–105, 188, 191 chemistry 92–3 complementary skills 93 finding 96–9 level of control 94 passion 93–4 red flags 102–3 successful matches 104–5 testing out 100–2 cohort analysis 237, 287–8 Color.com (social photo-sharing) app 113, 255 colour schemes 111 Commodore 20 communication open 412–13 team 194 with users 208–9 Companiesmadesimple.com 163–4 computers 20–1, 29 conferences 97–8, 202, 312–13 confidentiality provisions 244 connectedness 30 ConnectU 105 consumer audience apps 233–4 content, fresh 147 contracts 165–6 convertible loans 163 Cook, Daren 112 cookies 228–9 Coors 348 COOs see Chief Operations Officers Cost Per Acquisition (CPA) 148–9 Cost Per Download (CPD) 148 Costolo, Dick 77–8, 79–80 costs, and user acquisition 148–9, 184, 236–7, 275–9, 282 Crash Bandicoot 33 crawlers 146–7 Cray-1 supercomputer 20 CRM see customer-relationship management CrunchBase 238 CTOs see Chief Technology Officers Customer Acquisition Cost (CAC) 148–9, 184, 236–7, 275–9 customer lifecycle 212–14 customer segments 346–7 customer-centric approach 344 customer-relationship management (CRM) 290–4, 343 customer-support 208–9 Cutright, Alyssa 369 daily active users (DAUs) 142 D’Angelo, Adam 75–6 data 284–5, 345–7 data engineers 284 dating, online 14, 87–8, 101–2, 263 decision making 379–82, 407–8 defining apps 31–4 delegation 407 delight, delivery 205–7 design 82, 129–33, 206–7 responsive 144 designers 132, 189–91, 363, 376 developer meetups 97 developers see engineers/developers development see app development; software development development agencies 196 ‘development sprint’ 192 Devine, Rory 358–9 Digital Sky Technologies 385 directors of finance 316–17 Distimo 205 DLD 97 Doerr, John 164, 310 Doll, Evan 42–3, 105 domain names 109–10 international 146 protection 145–6 Domainnamesoup.com 109 Dorsey, Jack 41, 58, 72, 75–7, 79–80, 104, 112, 215–16, 305, 312, 412–13 ‘double-trigger’ vesting 247 DoubleClick 414 Dow Jones VentureSource 64 down rounds 322–3 downloads, driving 150–1 drag along rights 245 Dribbble.com 132 Dropbox 7, 90, 131, 276 CEO 407, 410–11 funding 160 scaling 336 staff 399 Dunbar, Robin 364–5 Dunbar number 365 e-commerce/marketplace 28–9, 67, 89, 213–14 Chinese 306 Flipboard and 351–2 and revenue engines 232, 233–4, 276 social media generated 271–2 and user retention 288, 289 eBay 7, 28–9, 131, 180, 276 economic models 275 economies of scale 331–2, 331–2 eCourier 15, 95 education 68–9 edX 69 Ek, Daniel 357 Ellis, Sean 182 emails 291–3 emotion effects of smartphones on 29–30, 30 inspiring 223–4 employees see staff employment contracts 246–7 engagement 236, 278, 283 engineering VPs 337, 358–9 engineers/developers 190–1, 194–5, 361–2, 362, 370, 375–7, 405 enterprise 90, 233–4 Entrepreneur First programme 160 entrepreneurs 3–5, 7–8, 65, 262, 393–4, 409, 424 Ericsson 21 Etsy 107, 109, 110, 358 Euclid Analytics 149 Evernote 7, 90, 131, 399 ExactTarget 291 excitement 30 executive assistants 367 Exitround 419 experience 67–8, 264, 397 Fab.com 352 Facebook 7, 10, 26, 32, 48, 76, 226, 394, 422 and acquisition of users 271, 272, 273–4 acquisitions 416–18, 417 agile culture 375 alerts 293 and application programming interface 36 board 180 and business identity 114 and Candy Crush 280–1 Chief Executive Officer 406 cofounders 100–1 and Color 255–6 design 131, 206, 363 Developer Garage 97 driving downloads on 151 and e-commerce decisions 271, 272 and FreeMyApps.com 271 funding 419 and getting your app found 147 and the ‘hacker way’ 375 initial public offering 420–1 and Instagram 29, 51, 76–80, 90, 117 name 110 ‘No-Meeting Wednesday’ 376 product development 187 profit per employee 403 revenue per employee 401 scaling 336 and Snapchat 57 staff 339, 362, 363, 398, 401, 403 and virality 281 WhatsApp purchase 42, 54–6, 416–17, 417 zero-user-acquisition cost 278 and Zynga 279, 281 Facetime app 46 fanatical users 294 feedback 86, 127, 134–5, 182, 192–3, 198–201, 256, 396 loops 204, 211 qualitative 199 quantitative 199 see also analytics Feld, Brad 170, 241 Fenwick and West 168 Fiksu 264, 269–70 finance, VP of 317–18 finding apps 140–8, 148–9 FireEye 90 First Data 419 first impressions 107–10 Fitbit 38 fitness bracelets 38 flat rounds 322–3 Flipboard 6, 29, 42–3, 49, 51, 89–90 and application programming interface 36 Catalogs 351–2 cofounders 105 design 131, 207 funding 164 growth 351–2 platform choice 119 product innovation 351–2 user notifications 292 virality 281 zero-user-acquisition cost 278 Flurry 135, 149, 220 Fontana, Ash 233 Forbes magazine 40 Ford Motors 419 Founder Institute, The 168 founder vesting 166–7, 244 Foursquare 419 France Telecom 13 franchising 354 FreeMyApps.com 270–1 Friedberg, David 412 Froyo (Android mobile software) 7 Fujii, Kiyotaka 304 full service agencies 195–6 functionality 25–6, 45–50, 131 funding 72, 75–6, 84, 87–8, 152–60, 161–71, 179 accelerators 159–60 angel investors 154, 155–6, 323 for billion-dollar apps 391 convertible loans 163 core documents 169–70 for five-hundred-million-dollar apps 328, 383–7 founder vesting 166–7 for hundred-million-dollar apps 254, 258, 316–17, 318–24 incubators 159–60 legal aspects 163–4 and revenue engines 233–4 Series A 234, 238–40, 238, 240, 241, 242–6, 255, 319–21, 385 Series B 238, 241, 253, 260, 284, 319–21, 322, 384 Series C 384 signing a deal 167–8 for ten-million-dollar apps 152–60, 161–71, 176, 235–49 venture capital 72, 75, 156–8, 165–6, 235–49, 261–2, 383–5, 385, 418–19 game in game 348–9 gaming 42, 47, 318, 355 business model 67, 89 and revenue engines 232, 278–9 and user retention 288, 289 see also specific games Gandhi, Sameer 336 Gartner 271 Gates, Bill 4 general managers (GMs) 300–3 Gladwell, Malcolm 424 Glassdoor 361–2 Global Positioning System (GPS) 23 Gmail 72 GMs see general managers goal setting 40–1, 310–11 Goldberg, Dave 397 Goldman Sachs 385 ‘good leavers’ 247 Google 7, 19, 23, 27, 72, 88, 164, 226 acquisitions 43, 414–16, 418 application programming interface 35–6 beta testing 202 Chief Executive Officer 406–8 developer meetups 97 finding your app on 144, 147 Hangouts app 46 meetings 381–2 mission 404, 408–9 and the OKR framework 310 profit per employee 403, 405 revenue per employee 401, 405 scaling 332 and Snapchat 57 and source attribution 228–9 staff 339, 340, 361–2, 366, 401, 403, 404–5, 412 Thank God It’s Friday (TGIF) meetings 311–12 transparency 413 value 78 Waze app purchase 43 and WhatsApp 56 zero-user-acquisition cost 278 see also Android (mobile operating system) Google Ad Mob 149 Google AdSense 149 Google Analytics 135, 219–20, 345 Google Glass 38–9, 405 Google I/O conference 313 Google Maps 33, 35, 414, 416 Google Now 37 Google Play 88, 89, 117, 120, 226 and beta testing 202 finding apps in 141–5 profit per employee 403 ratings plus comments 204–5 Google Reader 72 Google Ventures 384 Google X 405 Google+ and business identity 114 and virality 281 Google.org 339 GPS see Global Positioning System Graham, Paul 184–5, 211 Graphical User Interface (GUI) 20 Greylock 321, 383 Gross, Bill 406–7, 409–10 Groupon 7, 51–2, 227, 344–5, 419 Grove, Andy 310 growth 267, 308–17 buying sustained 417–18 engines 184, 210, 222–31, 259, 265 and five-hundred-million-dollar apps 329–36 and Friday update meetings 311–12 and goal setting 310–11 and hiring staff 308–9, 411–12 and product and development teams 313–14 and staff conferences 312–13 targets 234, 260 see also acquisition (of users); international growth; scaling Growth Hackers 182 GUI see Graphical User Interface hackathons 99 Haig, Patrick 143 Hailo app xiii–xiv, 5, 36, 89, 386 big data 284–5 branding 112–13 cofounders 94–6 customer segments 346–7 customer-support 208–9 design 131, 132, 133, 206–7 development 123–7, 153–4 Friday update meetings 311 funding 162, 242 goal setting 310 growth 296–7, 299, 302–4, 308–11, 313, 315–17, 329–30, 334–6 hiring staff 308–9, 334–6, 338, 366–7 idea for 14–18 international growth 296, 297, 299, 302–4 market research 182 marketing 263, 264, 268, 270, 273, 341, 347–8 meetings 381 metrics 137–9, 216 name 107 organisational culture 396 platform choice 117, 120, 121 premises xiii–xiv, 177–8, 329–30, 371–2, 386 product development 189, 191, 196 retention 293–4 revenue engine 276 scaling development and engineering 357 scaling people 365–7 scaling process 377 team 258 testing 177–8, 201–4 and user emotionality 224 virality 280, 282 Hangouts app 46 Harris Interactive 31 HasOffers 149 Hay Day 47, 97 head of data 342 Heads Up Display (HUD) 38 heart rate measurement devices 37–8 Hed, Niklas 42 hiring staff 308–9, 334–6, 337–40, 365–70 history of apps 31–2 HMS President xiii–xiv, 177–9, 329, 371, 386 HockeyApp 202 HootSuite 151 Houston, Drew 407, 410–11 HP 180, 402 HTC smartphone 121 HUD see Heads Up Display human universals 44–5 Humedica 419 hyperlinks 147 hypertext markup language (HTML) 147 I/O conference 2013 202 IAd mobile advertising platform 149 IBM 20, 402 icons 143 ideas see ‘thinking big’ identity of the business 86 branding 111–13 identity crises 106–14 names 106–11 websites 113–14 image descriptions 147 in Mobi 149 in-app purchases 28 incentive-based networks 270–1 incorporation 163–4, 179 incubators 159–60 Index Ventures 3, 261 initial public offerings (IPOs) 64, 67–9, 78, 80, 246, 420–2 innovation 404–5 Instagram 6, 29, 48, 51, 67, 71–80, 88–90, 114, 117, 226, 278, 340, 417–18 cofounders 73–4 design 131 funding 75–6, 77–8 X-Pro II 75 zero-user-acquisition cost 278 instant messaging 46 Instantdomainsearch.com 109 integrators 410 Intel 310 intellectual property 165–6, 244, 247 international growth 295–307 Angry Birds 297–9 Hailo 296, 297, 299, 302–4 language tools 297 Square 295, 299, 304–6 strings files 296 Uber 299–302 International Space Station 13 Internet bubble 13 investment see funding iOS software (Apple operating system) 7, 23–4, 46, 75, 104 advertising 274 audience size 119 building apps for 116–22 and international apps 296 scaling development and engineering 357–8 time spent on 26 iPad 42–3, 118–20, 351 iPhone 6, 19, 22–3, 32, 38–9, 183, 351 advertising on 274 camera 48 designing apps for 117–18, 120 finding apps with 145 games 42, 47, 58 and Instagram 74–6 in Japan 306 and Square 104, 306 and Uber 301 user spend 117 and WhatsApp app 54–5 iPod 22 IPOs see initial public offerings Isaacson, Walter 32 iTunes app 22, 47, 88, 143 iTunes U app 69 Ive, Jony 129 iZettle 304 Jackson, Eric 40 Jain, Ankit 142 Japan 227, 304–6 Jawbone Up 38 Jelly Bean (Android mobile software) 7 Jobs, Steve 4, 22, 32, 323, 393, 425 journalists 150–1 Jun, Lei 306 Kalanick, Travis 299–300, 384, 422 Kayak 336 Keret, Samuel 43 Keyhole Inc. 414 keywords 143, 146 Kidd, Greg 104 King.com 349, 421–2 see also Candy Crush Saga KISSmetrics 291 KitKat (Android mobile software) 7 Klein Perkins Caulfield Byers (KPCB) 158, 261, 321, 383 Kontagent 135 Koolen, Kees 320, 339 Korea 30 Koum, Jan 42, 54, 55–6, 154, 321, 394, 416 Kreiger, Mike 73–6 language tools 297 Launchrock.com 113–14, 145, 202 Lawee, David 415 lawyers 103, 169, 170, 242 leadership 410–11 see also Chief Executive Officers; managers lean companies 69, 115–22, 154, 257, 320–1 Lee, Bob 340 legalities 163–70, 242–7, 301 letting go 406–7 Levie, Aaron 396–7, 411 Levinson, Art 32 LeWeb 97 Libin, Phil 399 licensing 356 life experience 67–8, 264 lifetime value (LTV) 184, 215, 219, 220–1, 232, 275–7, 279, 291, 342 Line app 46, 226 Lingo24 297 LinkedIn 97, 226, 406, 408–9 links 147 liquidation preference 242, 243, 245 non-participating 245 Livio 419 loans, convertible 163 Localytics 135, 221 locations 69 logos 111–14 LTV see lifetime value luck 412 Luckey, Palmer 39 LVMH 304 Lyons, Carl 263 Maiden 95 makers 375–7 see also designers; engineers/developers managers 189–90, 300–3, 375–7, 405 MapMyFitness 419 market research 115, 127, 182 marketing data 345–7 and Facebook 271, 272, 273–4 and incentive-based networks 270–1 marketing engineering team 344–5 and mobile social media channels 271–3, 272 and mobile user-acquisition channels 269–70 partner marketing 347–8 scaling 341–9 teams 262–6, 337, 342 and traditional channels 268–9 VPs 262–6, 337, 342 marketplace see e-commerce/marketplace MasterCard 347–8 Matrix Partners 283 McClure, Dave 136, 160, 211, 234 McCue, Mike 42–3, 105, 351 McKelvey, Jim 41, 104 ‘me-too’ products 181 Medium 41 Meebo 73 meetings 379–82, 412–13 annual offsite 379 daily check-ins 381 disruptive nature 376–7 Friday update 311–12 meaningful 381–2 monthly strategic 380 quarterly 380 weekly tactical 380 Meetup.com 98–9 Mendelsen, Jason 170 messaging platforms 226 time spent on 46 and user retention 288, 289 metrics 136–9, 139, 211–21 activation 136, 137, 139, 153–4, 211–12, 213 annual revenue per user (ARPU) 215, 219, 232, 236 average transaction value (ATV) 214–15, 219, 232, 236, 387 consensual 215–16 lifetime value (LTV) 184, 215, 219, 220–1, 232, 275–7, 279, 291, 342 and product-market fit 209–10 referral 137, 138, 139, 153, 154, 211–12, 213, 230–1 revenue 137, 138, 139, 154, 211–12, 213, 214–15, 219, 291 transparency regarding 312 see also acquisition (of users); retention (of users) mice 20 Microsoft application programming interface 35–6 revenue per employee 401 Windows 20, 22, 24 Millennial Media 149 minimum viable product (MVP) 123, 153 MirCorp 13–14 mission 261, 404, 408–9 Mitchell, Jason 51 Mitsui Sumitomo Bank 305 Mixpanel.com tool 135, 217–18, 220–1, 287, 290–1, 345 MMS see Multimedia Messaging Service Mobile Almanac 45 Mobile App Tracking 230, 231 mobile technology, rise of 19–39 MoMo app 306 Monsanto 419 moonshots 404–5 Moore, Jonathan 200 MoPub 149 Moqups.com 128 Mosaic 180 Motorola 21 Moz.com 143 Mullins, Jacob 419 Multimedia Messaging Service (MMS) 47 Murphy, Bobby 43, 104–5, 152–3 music player apps 47 MVP see Metrics into Action; minimum viable product names 106–11, 142 NameStation.com 108 Nanigans 273–4 National Venture Capital Association 64 native apps 33–4 NDA see Non Disclosure Agreement negotiation 265 Net Promoter Score (NPS) 206, 209 net-adding users 206 Netflix 400 Netscape 164, 180 New Enterprise Associates 385 New York Times news app 32–3, 256 news and alerts feature 48–9 Nextstop 72 Nguyen, Bill 255–6 NHN 227 Nike Fuelband 38 Nintendo Game Boy 47 Nokia 21, 35–6 Non Disclosure Agreement (NDA) 165 noncompetition/non-solicitation provision 244, 247 notifications 291–4 NPS see Net Promoter Score Oculus VR 39 OKR (‘objectives and key results’) framework 310–11, 380 OmniGraffle 128 open-source software 23, 34–5, 185 OpenCourseWare 68–9 operating systems 20–4 see also Android; iOS software operations VPs 337 org charts 258, 309 organisational culture 395–8 O’Tierney, Tristan 104 outsourcing 194–7 ownership and founder vesting 166–7 and funding 155, 156, 161–3, 318 oxygen saturation measurement devices 37–8 Paananen, Ilkka 118–19, 397–8 Page, Larry 4, 23, 382, 404, 407–8 Palantir 90 Palihapitiya, Chamath 187 Pandora 7, 47, 67, 131, 410 pay-before-you-download model 28 pay-per-download (PPD) 225 Payleven 304 payment systems 7, 33–4, 227, 304, 305 see also Square app PayPal 7, 227, 304, 305 Pepsi 196 Perka 419 perks 398–400 perseverance 67, 394, 410 personal computers (PCs) 29 perspiration measurement devices 38 Pet Rescue Saga 349, 421 Petrov, Alex 369 phablets 7 Pham, Peter 255 PhoneSaber 33 Photoshop 128 PIN technology 305 Pincus, Mark 311 Pinterest app 48, 226 and business identity 114 and e-commerce decisions 271, 272 and getting your app found 147 name 107 and virality 281 Pishevar, Shervin 300 pivoting 73–4 population, global 9–10 portfolio companies 261–2 PowerPoint 128 PPD see pay-per-download preferential return 243 premises 370–2 preparation 412 press kits 148, 150 press releases 150 Preuss, Dom 98 privacy issues 43, 56–7 private vehicle hire see Uber pro-rata rights 242, 243 producers 409 product chunks 360 product development scaling 357–63 scope 199 team building for 188–91 and team location 193–4 and vision 186–8, 191 see also app development; testing product expansion 350–63 product extension 354 product managers 189–90, 405 product-centricity 185–6, 314, 360 product-market fit 9, 180–97, 235–6, 248, 256–7 measurement 209–10, 212, 286–8 profit 267, 320, 342 profit margin 258–9, 318, 321 profit per employee 402–4, 403, 405 profitability 260, 277, 400 Project Loon 405 proms 12 proto.io tool 133 prototype apps 86, 174 app Version 0.1 123–35, 174 new and improved Version 1.0 198–210 rapid-design prototyping 132–3 PRWeb 150 PSP 47 psychological effects of smartphones 29–30, 30 pttrns.com 131 public-relations agencies 343 publicity 150–1, 225, 313 putting metrics into action 138–9 Puzzles and Dragons 47, 131 QlikView 221, 284–5 QQ 307 quality assurance (QA) 190–1, 196 Quora 76 QZone 307 Rabois, Keith 368, 369 Rakuten 227 Rams, Dieter 129 rapid-design prototyping 132–3 ratings plus comments 204–5 Red Bull 223 redemption codes 230 referrals (user) 137, 138, 139, 153, 154, 211–12, 213 attribution for referrals 230–1 referral codes 230 religious apps 45 remuneration 361–2, 362, 363 Renault 13 restated certification 169 retention (of users) 136–9, 153, 154 for five hundred-million-dollar apps 327, 341–3 for hundred-million-dollar apps 286–94, 288–9 measurement 286–8 for ten-million-dollar apps 206, 211–12, 213, 278 revenue 137–8, 139, 154, 211–12, 213, 214–15, 219, 236, 239–40, 267, 291, 331–2, 341–2, 354 revenue engines 184, 210, 232–4, 257–66, 265, 275–85 revenue per employee 400–2, 402, 405 revenue streams 27–9 Ries, Eric, The Lean Startup 115–16 Rockefeller, John D. 9 Rocket Internet 304 Rolando 33 Rosenberg, Jonathan 413 Rovio 58, 97, 118, 297–9, 318, 320–1, 336, 354, 409 see also Angry Birds Rowghani, Ali 77 Rubin, Andy 23 Runa 419 SaaS see software as a service Sacca, Chris 75–6 sacrifice 86–7 Safari Web browser 32 salaries 361–2, 362, 363 sales VPs 337 Salesforce 291 Samsung 23 Galaxy Gear smartwatch 38 smartphones 121 Sandberg, Sheryl 4, 100–1, 339, 397 SAP 304 scaling 259, 308, 312, 323–4, 326, 330–6, 331–2, 384–5 decision making 379–81 international growth 295–307 marketing 341–9 and organisational culture 396–8 people 338–9, 364–72 premature 334–5 process 373–82 product development and engineering 357–63 and product innovation 350–6 reasons for 333–4 skill set for 335–6 Schmidt, Eric 120 scope 199 screenshots 131, 144, 206 scrum masters (‘agile coaches’) 315, 359, 360 search functions 49 organic 141–2, 141, 145 search-engine optimisation (SEO) 142, 145–8, 225 Sedo.com 109 Seed Fund 136 Seedcamp 160 Sega Game Gear 47 segmentation 220, 287, 290, 346–7 self-empowered squads/units 360 SEO see search-engine optimisation Sequoia Capital 76, 77–80, 158, 255, 321, 383, 385 Series A funding 234, 238–40, 238, 240, 241, 242–6, 255, 261, 262, 319–21, 385 Series B funding 238, 241, 253, 260, 319–21, 322, 384 Series C funding 384 Series Seed documents 168 Sesar, Steven 263 sex, smartphone use during 31 Shabtai, Ehud 43 shares 156, 166–8, 244 ‘sharing big’ 51–2, 52 Shinar, Amir 43 Shopzilla 263 Short Message Service (SMS) 21, 46–7 Silicon Valley 71–4, 77, 79, 99, 162, 168, 180, 184, 255, 340, 361, 411, 422 Sina 227 sitemaps 146–7 skills sets complementary 93 diverse 409–10 for scaling 335–6 Skok, David 283 Skype app 7, 46, 111, 200–1, 226, 357, 419 Sleep Cycle app 48 Smartling 297 smartwatches 7, 38–9 SMS see Short Message Service Snapchat app 6, 43, 46, 56–7, 88, 89, 223, 226, 416, 418 cofounders 104–5 design 131 funding 152–3, 307, 320 name 107 platform 117 staff 340 valuations 333 virality 280, 283 zero-user-acquisition cost 278 social magazines 42–3 see also Flipboard social media 48 driving downloads through 151 and getting your app found 147 mobile channels 271–3, 272 and user retention 288, 289 Sofa 363 SoftBank 227 software development agile 192–3, 299, 315, 357, 377 outsourcing 194–5 see also app development software as a service (SaaS) 67, 90, 208, 214, 233, 276–7 Somerset House 329–30, 371 Sony 21, 47 SoundCloud 358 source attribution 227–31 space tourism 13–14 speech-to-text technology 50 speed 20 Spiegel, Evan 43, 56–7, 104–5, 152–3 Spinvox 50 Splunk 90 Spotify app 47, 357–8 SQL 284 Square app 6, 41–2, 58–9, 87, 89, 333, 350 branding 112 Chief Executive Officer 412–13 cofounders 104 design 131, 363 funding 320–1 international growth 295, 299, 304–6 marketing 348 metrics 215–16 name 107, 110 product–market fit 183 revenue engine 276 scaling people 367–8 scaling product innovation 352–3 staff 340, 367–8 transparency 312 virality 282 Square Cash 353 Square Market 353 Square Register 350, 352–3 Square Wallet 348, 350, 353 Squareup.com 144 staff at billion-dollar app scale 395–405, 423 attracting the best 91 benefits 398–400 conferences 312–13 conflict 334, 378 employee agreements 244 employee legals 246–7 employee option pool 244 employee-feedback systems 378 firing 370, 378 hiring 308–9, 334–6, 411–12 induction programmes 370 investment in 360 mistakes 369–70, 411–12 and premises 370–2 profit per employee 402–4, 403 revenue per employee 400–2, 402 reviews 370 scaling people 364–72, 377–9 scrum masters 315, 359, 360 training programmes 370 see also cofounders; specific job roles; teams Staples 419 Starbucks 338, 348 startup weekends 98 startups, technology difficulties of building 63–80 failure 63–5, 73–4 identity 106–14 lean 115–22, 154 process 82–4, 85–105 secrets of success 66–9 step sensors 38 stock markets 420–1 straplines 111 strings files 296 Stripe 160 style 111 subscriptions 90 success, engines of 183–4, 423–4 SumUp 304 Supercell 28, 47, 97, 118–19, 318, 336, 397–8, 401, 403 see also Clash of Clans; Hay Day SurveyMonkey 397 surveys 206, 209 synapses 10 Systrom, Kevin 71–80 tablets 7 Tableau Software 90 Taleb, Nicholas Nassim 54 Tamir, Diana 51 Tap Tap Revolution (game) 42 Target 419 taxation 164 taxi hailing apps see Hailo app TaxiLight 16 team builders 264 team building 188–91 teams 82, 174, 252, 390 complementary people 409–10 for five-hundred-million-dollar apps 326, 342–5, 357–63, 374, 386 growth 313–14, 326, 342–4 for hundred-million-dollar apps 258–61 located in one place 193–4 marketing 262–6, 342–4 marketing engineering 344–5 product development and engineering 357–63 ‘two-pizza’ 374 TechCrunch Disrupt 97, 99 technology conferences 97–8, 202, 312–13 Techstars 159, 160, 168 Tencent 307 Tencent QQ 226 term sheets 168, 169, 170, 243–4 testing 126–8, 177–8, 187–8, 192–3, 199–201 beta 201–4 channels 224–7 text messaging 21 unlimited packages 42 see also Short Message Service ‘thinking big’ 40–59, 82, 85 big problem solutions 41–3 disruptive ideas 53–9 human universals 44–5 sharing big 51–2, 52 smartphones uses 45–50 Thoughtworks 196 time, spent checking smartphones 25–6, 26, 45–50 Tito, Dennis 13 tone of voice 111 top-down approaches 311 traction 233, 252 traffic information apps 43 traffic trackers 146 translation 296–7 transparency 311–12, 412–13 Trilogy 13 Tumblr 110, 226, 399, 418 Twitter 41, 48, 54, 72, 226, 394 acquisitions 418 and application programming interface 36 and Bootstrap 145 and business identity 114 delivering delight 206 and e-commerce decisions 272 and FreeMyApps.com 271 funding 419, 421 and getting your app found 147 initial public offering 421 and Instagram 51, 76–7, 79–80 name 110 and virality 281 ‘two-pizza’ teams 374 Uber 6, 36, 87, 89, 333, 350 and attribution for referrals 231 design 131 funding 320, 384, 422 international growth 295, 299–302 name 107, 110 revenue engine 276 revenue per employee 401 scaling product innovation 355–6 staff 339, 399 user notifications 292 virality 280 Under Armour 419 Union Square Ventures (USV) 3, 158, 242, 261, 262, 288, 321, 323, 377, 383 unique propositions 198 UnitedHealth Group 419 URLs 110 ‘user experience’ (UX) experts 190 user journeys 127–8, 213–14 user notifications 291–4 user stories 193 users 83, 175, 252, 327, 390 activation 136, 137, 139, 153–4, 211–12, 213 annual revenue per user (ARPU) 215, 219, 232, 236 communication with 208–9 definition 137 emotional response of 223–4 fanatical 294 finding apps 140–8 lifetime value (LTV) 184, 215, 219, 220–1, 232, 275–7, 279, 291 metrics 136–9 net-adding of 206 ratings plus comments 204–5 referrals 137, 138, 139, 153, 154, 211–12, 213, 230–1 target 83, 115, 127 wants 180–97 see also acquisition (of users); retention (of users) Usertesting.com 200–1 USV see Union Square Ventures valuations 83, 161–3, 175, 237–8, 238, 253, 318, 319, 322, 327, 333, 391 venture capital 72, 75, 156–8, 165–6, 235–49, 261–2, 383–5, 385, 418–19 Viber app 6, 46, 1341 video calls 46, 47 viral coefficient 282–4 ‘viral’ growth 225, 278, 279–84 Communication virality 281 and cycle time 283–4 incentivised virality 280–1 inherent virality 280 measurement 282–4 social-network virality 281 word-of-mouth virality 281–2 virtual reality 39 vision 261, 393–4, 408–9, 414, 415 voice calls 46–7 voice-over-Internet protocol (VOIP) 46 voicemail 50 Wall Street Journal 43, 55 warranties 246 Waze app 6, 43, 97 acquisition 415–16 design 131 name 107 zero-user-acquisition cost 278 web browsing 49 Web Summit 97 websites 113–14, 144–8 WebTranslateIt (WTI) 297 WeChat app 46, 226, 306 Weibo 48 Weiner, Jeff 408–9 Wellington Partners 4 Weskamp, Marcos 207 Westergren, Tim 410 WhatsApp 6, 42, 46, 54–6, 87, 90, 226, 394 acquisition 42, 54–6, 416, 416–17, 417 cofounders 96 design 131, 144 funding 154, 320–1 platform 117–18 valuations 333 virality 280 White, Emily 340 Williams, Evan 41, 65 Williams, Rich 344 Wilson, Fred 110, 242, 288, 323, 377 Windows (Microsoft) 20–1, 22, 24, 24 Winklevoss twins 105 wireframes 127–8 Woolley, Caspar 15–16, 95, 124, 338 WooMe.com 14, 87–8, 101–2, 263 Workday 90 world population 9–10 Worldwide Developers Conference (WWDC) 313 wowing people 8–9 WTI see WebTranslateIt Xiaomi 306 Y Combinator 159–60, 184–5, 211, 407, 410–11 Yahoo!


pages: 933 words: 205,691

Hadoop: The Definitive Guide by Tom White

Amazon Web Services, bioinformatics, business intelligence, combinatorial explosion, database schema, Debian, domain-specific language, en.wikipedia.org, fault tolerance, full text search, Grace Hopper, information retrieval, Internet Archive, Kickstarter, linked data, loose coupling, openstreetmap, recommendation engine, RFID, SETI@home, social graph, web application

Machine logs, RFID readers, sensor networks, vehicle GPS traces, retail transactions—all of these contribute to the growing mountain of data. The volume of data being made publicly available increases every year, too. Organizations no longer have to merely manage their own data: success in the future will be dictated to a large extent by their ability to extract value from other organizations’ data. Initiatives such as Public Data Sets on Amazon Web Services, Infochimps.org, and theinfo.org exist to foster the “information commons,” where data can be freely (or in the case of AWS, for a modest price) shared for anyone to download and analyze. Mashups between different information sources make for unexpected and hitherto unimaginable applications. Take, for example, the Astrometry.net project, which watches the Astrometry group on Flickr for new photos of the night sky.

Setup First install Whirr by downloading a recent release tarball, and unpacking it on the machine you want to launch the cluster from, as follows: % tar xzf whirr-x.y.z.tar.gz Whirr uses SSH to communicate with machines running in the cloud, so it’s a good idea to generate an SSH keypair for exclusive use with Whirr. Here we create an RSA keypair with an empty passphrase, stored in a file called id_rsa_whirr in the current user’s .ssh directory: % ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa_whirr Warning Do not confuse the Whirr SSH keypair with any certificates, private keys, or SSH keypairs associated with your Amazon Web Services account. Whirr is designed to work with many cloud providers, and it must have access to both the public and private SSH key of a passphrase-less keypair that’s read from the local filesystem. In practice, it’s simplest to generate a new keypair for Whirr, as we did here. We need to tell Whirr our cloud provider credentials. We can export them as environment variables as follows, although you can alternatively specify them on the command line, or in the configuration file for the service

These events are first filtered and processed, and then handed to various backend systems, including AsterData, Hypertable, and Katta. The volume of these events can be huge, too large to process with traditional systems. This data can also be very “dirty” thanks to “injection attacks” from rogue systems, browser bugs, or faulty widgets. For this reason, ShareThis chose to deploy Hadoop as the preprocessing and orchestration frontend to their backend systems. They also chose to use Amazon Web Services to host their servers, on the Elastic Computing Cloud (EC2), and provide long-term storage, on the Simple Storage Service (S3), with an eye toward leveraging Elastic MapReduce (EMR). In this overview, we will focus on the “log processing pipeline” (Figure 16-19). The log processing pipeline simply takes data stored in an S3 bucket, processes it (described shortly), and stores the results back into another bucket.


pages: 192 words: 44,789

Vagrant: Up and Running by Mitchell Hashimoto

Amazon Web Services, barriers to entry, Debian, DevOps, remote working, software as a service, web application

With just two commands and zero configuration, Vagrant automatically builds and configures a fully featured virtual machine for any purpose. And with just a little bit of easy-to-learn configuration, Vagrant can automatically set up complex network configurations, install and manage software within the virtual machine, or package the virtual machine for re-use by other people. Virtualization is the foundational technology behind what is often referred to as the cloud. Amazon Web Services, Microsoft Azure, virtual private server (VPS) providers, and more are based completely around this technology or those similar to it. These sort of cloud services are now the de facto standard for hosting web applications. Virtualization is everywhere. The good news is that virtualization technology is readily available to anyone with a modern computer. The bad news is that we’re only at the tip of the iceberg of what is possible with this technology.


pages: 464 words: 127,283

Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia by Anthony M. Townsend

1960s counterculture, 4chan, A Pattern Language, Airbnb, Amazon Web Services, anti-communist, Apple II, Bay Area Rapid Transit, Burning Man, business process, call centre, carbon footprint, charter city, chief data officer, clean water, cleantech, cloud computing, computer age, congestion charging, connected car, crack epidemic, crowdsourcing, DARPA: Urban Challenge, data acquisition, Deng Xiaoping, digital map, Donald Davies, East Village, Edward Glaeser, game design, garden city movement, Geoffrey West, Santa Fe Institute, George Gilder, ghettoisation, global supply chain, Grace Hopper, Haight Ashbury, Hedy Lamarr / George Antheil, hive mind, Howard Rheingold, interchangeable parts, Internet Archive, Internet of things, Jacquard loom, Jane Jacobs, jitney, John Snow's cholera map, Joi Ito, Khan Academy, Kibera, Kickstarter, knowledge worker, load shedding, M-Pesa, Mark Zuckerberg, megacity, mobile money, mutually assured destruction, new economy, New Urbanism, Norbert Wiener, Occupy movement, off grid, openstreetmap, packet switching, Panopticon Jeremy Bentham, Parag Khanna, patent troll, Pearl River Delta, place-making, planetary scale, popular electronics, RFC: Request For Comment, RFID, ride hailing / ride sharing, Robert Gordon, self-driving car, sharing economy, Silicon Valley, Skype, smart cities, smart grid, smart meter, social graph, social software, social web, special economic zone, Steve Jobs, Steve Wozniak, Stuxnet, supply-chain management, technoutopianism, Ted Kaczynski, telepresence, The Death and Life of Great American Cities, too big to fail, trade route, Tyler Cowen: Great Stagnation, undersea cable, Upton Sinclair, uranium enrichment, urban decay, urban planning, urban renewal, Vannevar Bush, working poor, working-age population, X Prize, Y2K, zero day, Zipcar

During a scary but not very destructive earthquake on the US East Coast in the summer of 2011, cell networks were again overwhelmed. Yet media reports barely noted it. Cellular outages during crises have become so commonplace in modern urban life that we no longer question why they happen or how the problem can be fixed. Disruptions in public cloud-computing infrastructure highlight the vulnerabilities of dependence on network apps. Amazon Web Services, the eight-hundred-pound gorilla of public clouds that powers thousands of popular websites, experienced a major disruption in April 2011, lasting three days. According to a detailed report on the incident posted to the company’s website, the outage appears to have been a normal accident, to use Perrow’s term. A botched configuration change in the data center’s internal network, which had been intended to upgrade its capacity, shunted the entire facility’s traffic onto a lower-capacity backup network.

., 265 ACM Queue, 266 Adams, Sam, 83 Aerotropolis (Kasarda and Lindsay), 24 Agent.btz, 269 Airbnb, 163 air-conditioning, early solutions for, 19–20 air defense, computer systems for, 63 Air Force, U.S., 63, 259 “air-gapping,” 269 AirPort, 128 air transportation, 63 digital technology in, 32–33 Albritton, Dan, 301–2 Alexander, Christopher, 142–44, 285–86 Alfeld, Louis Edward, 81–82, 86 Allan, Alasdair, 271 Altair, MITS, 153 Altman, Anne, 65 Amar, Georges, 106, 133 Amazon Web Services, 263–64 American Airlines, 63–64 American Express, 62 Amin, Massoud, 35 Amsterdam, 279 analog cellular, 53 Angelini, Alessandro, 91–92 Ansari X PRIZE, 202–3 API (application program interface), 150 Apple, 49, 128, 148, 271 Siri of, 233 apps, 121–26, 144–52, 183, 213, 235 to address urban problems, 156–59 badges for, 148 contests for, 156, 200–205, 212, 215, 225, 227–30 for navigation of disabled, 166 situated software as, 232–36 “Trees Near You” as, 201–2 variety of, 6 Apps for Democracy, 156, 200–201, 203 Arab Spring, social media in, 11–12 Arbon, 37 Arcaute, Elsa, 313–14 Archibald, Rae, 80 Archigram, 20–21 Architectural Association (London), 20 Architectural Forum, 142 Architecture Without Architects (Rudofsky), 111–12 Arduino, 137–41 ARPA (Advanced Research Projects Agency), 259 ARPANET, 111, 259–60, 269 ArrivalStar, 293 Arup, 32 Ashlock, Philip, 158–59 Asimov, Isaac, 73–75, 88 Association for Computing Machinery, 260 Astando, 244 AT&T, 35–37, 51–52, 111, 260, 272 dial-up Internet service at, 36 Atlanta, Ga., 66 Atlantic, The, 75 AutoCAD, 302 AutoDesk, 302 automobile, as new technology, 7 Ayers, Charlie, 252 Babajob, 178–79 “Baby Bells,” 195 Baltimore, Md., 211 Banavar, Guru, 66–67, 69, 90, 306 Bangalore, 66, 178–79 Cisco’s smart city engineering group at, 45 as fast-growing city, 13 Ban Ki-moon, 181–82 Banzi, Massimo, 137 Baran, Paul, 259–60 Barcelona, 10, 246–47 destruction of wall of, 43 Barragán, Hernando, 137 Barry, Marion, 199 Batty, Michael, 85–87, 295–97, 313, 315–16 Becker, Gene, 112–13 Beijing, 49, 273–74 Belloch, Juan Alberto, 223 Beniger, James, 42–43 Bentham, Jeremy, prison design of, 13 Berlin, 38 Bernstein, Phil, 302 Bettencourt, Luis, 312–13 Betty, Garry, 196 Bhoomi, 12–13 big data, 29, 87, 191, 292–93, 297, 305–6, 316, 319 “Big Ideas from Small Places” (Khanna and Skilling), 224 BlackBerry Messenger, riots coordinated via, 12 blogosphere, 155 Bloomberg, Michael, 147, 205–6, 304 Boing-Boing, 156 Booz Allen Hamilton, 30 Bosack, Len, 44 Boston, Mass., 212–17, 239–41, 306–7 “Adopt-A-Hydrant” in, 213 Discover BPS, 240–42 Office of New Urban Mechanics in, 213–16 “What Are My Schools?”


pages: 212 words: 49,544

WikiLeaks and the Age of Transparency by Micah L. Sifry

1960s counterculture, Amazon Web Services, banking crisis, barriers to entry, Bernie Sanders, Buckminster Fuller, Chelsea Manning, citizen journalism, Climategate, crowdsourcing, Google Earth, Howard Rheingold, Internet Archive, Jacob Appelbaum, John Markoff, Julian Assange, Network effects, RAND corporation, school vouchers, Skype, social web, source of truth, Stewart Brand, web application, WikiLeaks

SIFRY 13 14 15 16 17 18 19 20 21 22 23 24 205 January 28, 2011, www.reuters.com/article/2011/01/28/us-wikileaksidUSTRE70R5A120110128?pageNumber=1. See http://about.lob.by/localeaks/ for details. Micah L. Sifry, “From WikiLeaks to OpenLeaks, Via the Knight News Challenge,” techPresident, December 17, 2010, http://techpresident.com/ blog-entry/wikileaks-openleaks-knight-news-challenge. Amazon Web Services, http://aws.amazon.com/message/65348. Interview with author, December 1, 2010. PayPal’s vice president of platform, Osama Bedier, said that the company acted in response to the State Department legal advisor’s letter to WikiLeaks, which declared its receipt of the leaked cables to be a violation of the law. PayPal’s terms of service says its payment system “cannot be used for any activities that encourage, promote, facilitate or instruct others to engage in illegal activity.”


Learning Puppet 4: A Guide to Configuration Management and Automation by Jo Rhett

Amazon Web Services, Debian, DevOps, Golden Gate Park, pull request

You’ll keep this environment long after you have finished this book. If you are an experienced developer or operations engineer, you are welcome to use a testing environment of your own choice. Anything which can host multiple Linux nodes will work. Puppet’s needs are minimal. Any of the following would be suitable for use as a Puppet test lab: • • • • • A bunch of spare systems you have sitting around that you can install Linux on. An AWS Free Tier Amazon Web Services instance. An OpenStack DevStack development instance. An VMware Free vSphere ESXi solo instance. A Vagrant development environment on your personal computer. You can build your own test lab using one of the solutions above, or you can use an existing test lab you maintain. In all cases I recommend using an OS compatible with RedHat Enterprise Linux 6 or 7 for learning purposes. The CentOS platform is freely available, and fully supported by both Red Hat and Puppet Labs.


pages: 186 words: 49,251

The Automatic Customer: Creating a Subscription Business in Any Industry by John Warrillow

Airbnb, airport security, Amazon Web Services, asset allocation, barriers to entry, call centre, cloud computing, commoditize, David Heinemeier Hansson, discounted cash flows, high net worth, Jeff Bezos, Network effects, passive income, rolodex, sharing economy, side project, Silicon Valley, Silicon Valley startup, software as a service, statistical model, Steve Jobs, Stewart Brand, subscription business, telemarketer, time value of money, zero-sum game, Zipcar

Not surprisingly, its first focus was on baby products like diapers and wipes—a category Amazon placed a big bet on when it paid $545 million to acquire Quidsi, the creators of Diapers.com, which itself offers a subscription for diapers that enjoyed 30% month-over-month growth in 2013.6 Amazon is known for its wins in selling to consumers—but subscriptions can work for B2B as well as B2C. One of Amazon’s latest ventures is a subscription that offers to help other companies grow their subscription businesses. Amazon Web Services (AWS) offers companies access to servers, software, and technology support on a subscription basis. Many of the world’s largest subscription companies, including Adobe, Citrix, Netflix, and Sage, use AWS, along with many of the highest-profile start-ups, like Airbnb, Pinterest, Dropbox, and Spotify. Amazon is pioneering the subscription model in virtually every area of its business, but the subscription model is nothing new.


pages: 271 words: 52,814

Blockchain: Blueprint for a New Economy by Melanie Swan

23andMe, Airbnb, altcoin, Amazon Web Services, asset allocation, banking crisis, basic income, bioinformatics, bitcoin, blockchain, capital controls, cellular automata, central bank independence, clean water, cloud computing, collaborative editing, Conway's Game of Life, crowdsourcing, cryptocurrency, disintermediation, Edward Snowden, en.wikipedia.org, Ethereum, ethereum blockchain, fault tolerance, fiat currency, financial innovation, Firefox, friendly AI, Hernando de Soto, intangible asset, Internet Archive, Internet of things, Khan Academy, Kickstarter, lifelogging, litecoin, Lyft, M-Pesa, microbiome, Network effects, new economy, peer-to-peer, peer-to-peer lending, peer-to-peer model, personalized medicine, post scarcity, prediction markets, QR code, ride hailing / ride sharing, Satoshi Nakamoto, Search for Extraterrestrial Intelligence, SETI@home, sharing economy, Skype, smart cities, smart contracts, smart grid, software as a service, technological singularity, Turing complete, uber lyft, unbanked and underbanked, underbanked, web application, WikiLeaks

Genomecoin, GenomicResearchcoin Even without considering the longer-term speculative possibilities of the complete invention of an industrial-scale all-human genome sequencing project with the blockchain, just adding blockchain technology as a feature to existing sequencing activities could be enabling. Conceptually, this would be like adding coin functionality or blockchain functionality to services like DNAnexus, a whole-human genome cloud-based storage service. Operating in collaboration with university collaborators (Baylor College of Medicine’s Human Genome Sequencing Center) and Amazon Web Services, the DNAnexus solution is perhaps the largest current data store of genomes, having 3,751 whole human genomes and 10,771 exomes (440 terabytes) as of 2013.147 The progress to date is producing a repository of 4,000 human genomes, out of the possible field of 7 billion humans, which highlights the need for large-scale models in these kinds of big data projects (human whole-genome sequencing).


pages: 215 words: 56,215

The Second Intelligent Species: How Humans Will Become as Irrelevant as Cockroaches by Marshall Brain

Amazon Web Services, basic income, clean water, cloud computing, computer vision, digital map, en.wikipedia.org, full employment, income inequality, job automation, knowledge worker, low earth orbit, mutually assured destruction, Occupy movement, Search for Extraterrestrial Intelligence, self-driving car, Stephen Hawking, working poor

No one in 2000 would have imagined a company like Google with control over millions of server machines and petabytes of storage space. Yet here we are in 2015 and Google exists in that form. Many other companies – Microsoft, Facebook, Apple, Amazon, etc. - have similar amounts of computing power and storage space. And Amazon makes it possible for anyone to easily build their own cloud platform using a system called AWS (Amazon Web Services). The processing power, hard disk space and RAM in a typical desktop computer has increased dramatically because of Moore's Law since desktop machines first appeared in the 1980s. Extrapolating out to the years 2020 and 2040 shows a startling increase in computer power. The point where small, inexpensive computers have power approaching that of the human brain is just a few decades away.


pages: 499 words: 144,278

Coders: The Making of a New Tribe and the Remaking of the World by Clive Thompson

2013 Report for America's Infrastructure - American Society of Civil Engineers - 19 March 2013, 4chan, 8-hour work day, Ada Lovelace, AI winter, Airbnb, Amazon Web Services, Asperger Syndrome, augmented reality, Ayatollah Khomeini, barriers to entry, basic income, Bernie Sanders, bitcoin, blockchain, blue-collar work, Brewster Kahle, Brian Krebs, Broken windows theory, call centre, cellular automata, Chelsea Manning, clean water, cloud computing, cognitive dissonance, computer vision, Conway's Game of Life, crowdsourcing, cryptocurrency, Danny Hillis, David Heinemeier Hansson, don't be evil, don't repeat yourself, Donald Trump, dumpster diving, Edward Snowden, Elon Musk, Erik Brynjolfsson, Ernest Rutherford, Ethereum, ethereum blockchain, Firefox, Frederick Winslow Taylor, game design, glass ceiling, Golden Gate Park, Google Hangouts, Google X / Alphabet X, Grace Hopper, Guido van Rossum, Hacker Ethic, HyperCard, illegal immigration, ImageNet competition, Internet Archive, Internet of things, Jane Jacobs, John Markoff, Jony Ive, Julian Assange, Kickstarter, Larry Wall, lone genius, Lyft, Marc Andreessen, Mark Shuttleworth, Mark Zuckerberg, Menlo Park, microservices, Minecraft, move fast and break things, move fast and break things, Nate Silver, Network effects, neurotypical, Nicholas Carr, Oculus Rift, PageRank, pattern recognition, Paul Graham, paypal mafia, Peter Thiel, pink-collar, planetary scale, profit motive, ransomware, recommendation engine, Richard Stallman, ride hailing / ride sharing, Rubik’s Cube, Ruby on Rails, Sam Altman, Satoshi Nakamoto, Saturday Night Live, self-driving car, side project, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, single-payer health, Skype, smart contracts, Snapchat, social software, software is eating the world, sorting algorithm, South of Market, San Francisco, speech recognition, Steve Wozniak, Steven Levy, TaskRabbit, the High Line, Travis Kalanick, Uber and Lyft, Uber for X, uber lyft, universal basic income, urban planning, Wall-E, Watson beat the top human players on Jeopardy!, WikiLeaks, women in the workforce, Y Combinator, Zimmermann PGP, éminence grise

if (sys.platform.startswith(“linux”)): That one tiny missing colon breaks the program. “See, this is what I’m talking about,” Spectre says, slapping his laptop shut with a grimace. “The distance between looking like a genius and looking like an idiot in programming? It’s one character wide.” Even the slightest mistake in an instruction can produce catastrophe. One morning in early 2017, there was a massive collapse of Amazon Web Services, a huge cloud-computing system used by thousands of web apps, including some huge ones, like Quora or Trello. For over three hours, many of those big internet services were impaired. When Amazon finally got things up and running again and sent in a team to try and figure out what had gone wrong, it appeared that the catastrophe had emerged from one of their systems engineers making a single mistyped command.

See artificial intelligence (AI) Albright, Jonathan, ref1 Alciné, Jacky, ref1 algorithms, ref1, ref2 bias in ranking systems, ref1 scale and, ref1 algorithms challenge whiteboard interview, ref1, ref2, ref3 Algorithms of Oppression (Noble), ref1 Allen, Fran, ref1, ref2 Allen, Paul, ref1 AlphaGo, ref1, ref2 Altman, Sam, ref1, ref2 Amabile, Teresa M., ref1 Amazon, ref1, ref2, ref3 Amazons (board game), ref1 Amazon Web Services, ref1 Analytical Engine, ref1 Anderson, Tom, ref1 AND gate, ref1 Andreessen, Marc, ref1, ref2, ref3, ref4, ref5, ref6, ref7, ref8 Antisocial Media (Vaidhyanathan), ref1 Apple, ref1 Apple I, ref1 Apple iPhone, ref1, ref2 aptitude testing, ref1 architects, ref1 artificial intelligence (AI), ref1 dangers of, warnings about and debate over, ref1 de-biasing of, ref1 deep learning (See deep learning) edge cases and, ref1 expert systems, ref1 Hollywood depiction of, ref1 initial attempts to create, at Dartmouth in 1956, ref1 job listing sites, biased results in, ref1 justice system, effect of AI bias on, ref1 learning problem, ref1 neural nets (See neural nets) racism and sexism, learning of, ref1 artistic temperaments, ref1 Assembly computer language, ref1 Atwood, Jeff, ref1, ref2 Babbage, Charles, ref1, ref2 back-end code, ref1, ref2, ref3, ref4 backpropagation, ref1 “Bad Smells in Code” (Fowler and Beck), ref1 Baffler, The, ref1 Bahnken, A.


We Are the Nerds: The Birth and Tumultuous Life of Reddit, the Internet's Culture Laboratory by Christine Lagorio-Chafkin

4chan, Airbnb, Amazon Web Services, Bernie Sanders, big-box store, bitcoin, blockchain, Brewster Kahle, Burning Man, crowdsourcing, cryptocurrency, David Heinemeier Hansson, Donald Trump, East Village, game design, Golden Gate Park, hiring and firing, Internet Archive, Jacob Appelbaum, Jeff Bezos, jimmy wales, Joi Ito, Justin.tv, Kickstarter, Lean Startup, Lyft, Marc Andreessen, Mark Zuckerberg, medical residency, minimum viable product, natural language processing, Paul Buchheit, Paul Graham, paypal mafia, Peter Thiel, plutocrats, Plutocrats, QR code, recommendation engine, RFID, rolodex, Ruby on Rails, Sam Altman, Sand Hill Road, Saturday Night Live, self-driving car, semantic web, side project, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, slashdot, Snapchat, social web, South of Market, San Francisco, Startup school, Stephen Hawking, Steve Jobs, Steve Wozniak, technoutopianism, uber lyft, web application, WikiLeaks, Y Combinator

He took one of the old URLs he and Huffman had stashed away because they were too funny to let expire, Breadpig.com, and began building what would become a nonprofit to support independent creative projects. He’d go out with a bang, at least. They’d throw a huge Halloween party for loyal Redditors that night at a sports bar in Potrero Hill called the Connecticut Yankee. Before the party, they’d accomplish a bit of hardware handiwork they’d begun that past spring. Back in mid-May, the company had transitioned all its operations and the site’s storage to Amazon Web Services. It had been a bit bumpy at first—the team pulled an all-nighter to figure out how to fix the site’s performance when its servers weren’t sitting inches from one another, until Huffman coded a fix. They’d kept the old servers around in case of trouble, but five months had passed without significant incident. So for Halloween, finally, they’d physically decommission them. The five guys loaded into Schiraldi’s gray Toyota Yaris and went the few blocks over from 3rd Street to Spear and Harrison, to the ColoServe facility where Reddit’s servers were racked.

The server’s name: Servfefe. Other recent names include ServyMcServface, fidgetspinner, IllBeYourServerTonight, Hillary-email-server, and SeanSpicer-porn-folder. “The way our servers work, they get spun up and killed quite a lot,” Wardle said. “So probably for a brief period, yes, we had one called Snowden. There was definitely one called Harambe.” Reddit’s operations were managed across Amazon Web Services and employed between six hundred and one thousand servers at a time, scaling to more during busy times of day. There was no physical AWS server bestowed with a silver nameplate thanks to a vote of random humans scattered across the planet, so none of this makes a tremendous amount of sense. But damn if it isn’t funny. * * * When Wong pulled his ocean-blue Tesla onto Pioneer Way, a cul-de-sac in downtown Mountain View, in the spring of 2014, he saw several open parking spots in the lot to the side of Y Combinator’s headquarters.


pages: 201 words: 63,192

Graph Databases by Ian Robinson, Jim Webber, Emil Eifrem

Amazon Web Services, anti-pattern, bioinformatics, commoditize, corporate governance, create, read, update, delete, data acquisition, en.wikipedia.org, fault tolerance, linked data, loose coupling, Network effects, recommendation engine, semantic web, sentiment analysis, social graph, software as a service, SPARQL, web application

With this strategy, writes to the cluster are buffered in a queue; a worker then polls the queue and executes batches of writes against the database. Not only does this regulate write traffic, it reduces contention, and allows you to pause write operations without refusing client requests during maintenance periods. Global clusters For applications catering to a global audience, it is possible to install a multi-region cluster in multiple datacenters and on cloud platforms such as Amazon Web Services 7. See http://docs.neo4j.org/chunked/milestone/ha-configuration.html Application Architecture | 73 (AWS). A multi-region cluster allows you to service reads from the portion of the cluster geographically closest to the client. In these situations, however, the latency introduced by the physical separation of the regions can sometimes disrupt the coordination pro‐ tocol; it is, therefore, often desirable to restrict master reelection to a single region.


pages: 285 words: 58,517

The Network Imperative: How to Survive and Grow in the Age of Digital Business Models by Barry Libert, Megan Beck

active measures, Airbnb, Amazon Web Services, asset allocation, autonomous vehicles, big data - Walmart - Pop Tarts, business intelligence, call centre, Clayton Christensen, cloud computing, commoditize, crowdsourcing, disintermediation, diversification, Douglas Engelbart, Douglas Engelbart, future of work, Google Glasses, Google X / Alphabet X, Infrastructure as a Service, intangible asset, Internet of things, invention of writing, inventory management, iterative process, Jeff Bezos, job satisfaction, Kevin Kelly, Kickstarter, late fees, Lyft, Mark Zuckerberg, Oculus Rift, pirate software, ride hailing / ride sharing, self-driving car, sharing economy, Silicon Valley, Silicon Valley startup, six sigma, software as a service, software patent, Steve Jobs, subscription business, TaskRabbit, Travis Kalanick, uber lyft, Wall-E, women in the workforce, Zipcar

Amazon believes that it will someday be able to predict what its customers want so accurately that it will begin the shipping process before the orders are even placed. If you haven’t heard, this deep data strategy is paying off. Amazon has one of the best customer satisfaction ratings in the United States, and it translates that wealth of customer data into new customer-satisfying innovations like the Kindle, Amazon streaming, and Amazon Web Services. Maintaining specific, measurable goals and tracking key metrics, leaders at Amazon can make decisions quickly and confidently, with the best information possible. The rapid iteration required of digital innovation—particularly for network businesses—requires this type of support. PIVOT Step 5: Track The goal of Track, an ongoing process, is to determine which metrics and which reporting frequency you need to best support the development of your network business.


pages: 982 words: 221,145

Ajax: The Definitive Guide by Anthony T. Holdener

AltaVista, Amazon Web Services, business process, centre right, create, read, update, delete, database schema, David Heinemeier Hansson, en.wikipedia.org, Firefox, full text search, game design, general-purpose programming language, Guido van Rossum, information retrieval, loose coupling, MVC pattern, Necker cube, p-value, Ruby on Rails, slashdot, sorting algorithm, web application

Following these syntax rules, the basic skeleton for SOAP looks like this: <?xml version="1.0" encoding="utf-8"?> <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope" soap: encodingStyle="http://www.w3.org/2001/12/soap-encoding"> <soap:Header> <!-- Header information --> </soap:Header> <soap:Body> <!-- Body Information --> </soap:Body> </soap:Envelope> Example 17-1 shows what a request may look like using Amazon Web Services (AWS) to get details regarding this book. The Amazon Standard Item Number (ASIN) is how Amazon tracks every item that it sells. In the case of books, the ASIN is the same as the book’s ISBN. Example 17-1. A SOAP request using AWS <?xml version="1.0" encoding="utf-8" ?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"> <SOAP-ENV:Body> 598 | Chapter 17: Introducing Web Services Example 17-1.

Send email to index@oreilly.com. 925 Ajax web applications (see RIAs) Ajax.NET, 97 Ajax.PeriodicalUpdater() object, 690, 868 Ajax.PeriodicUpdater object, 101 Ajax.Request object, 99 Ajax.Request( ) method, callbacks, 416 Ajax.Responders object, 867 Ajax.Updater object, 101, 867 options, 101 ajaxload.info web site, 436 Akismet (blogging API), 620, 892 alert boxes, 335 integrating the window into web applications, 335–347 keeping focus and closing the pop up, 339–343 moving the window, 344–347 window style, 336–339 variations caused by operating systems and browsers, 335 alpha channel, 438 alpha transparency, 438 support by PNG format, 440 alternate stylesheets, 364, 371 font size, 389 “Alternative Style: Working with Alternate Style Sheets”, 372 Amazon Standard Item Number (ASIN), 598 Amazon Web Services (see AWS) Amazon.com Book page, 154 breadcrumbs on, 221 organizing tools, 155 web services, 652 XML Scratch Pad, 609 Amnesty International, 656, 892 Amp’s main page, 144 animation, 434–481 Ajax, 453–481 dragging and dropping, 454–464 drawing libraries, 472–481 dynamically animating position of an object, 464–467 effects on objects, 467–472 object manipulations, 467–472 using frameworks, 453 building with PNG format, 439–453 CSS, 441 differences from GIFs, 440 JavaScript looping, 442–444 926 | Index more robust animation object, 444–448 using Ajax, 448–453 character animation for games, 735–753 moving the character, 742–753 walking loop, 735–742 GIF format history of, 435 how it works, 435–437 on the Web, 434 animation object (example), 442–444 adding Ajax, 448–453 starting, pausing, and stopping programmatically, 444–448 AOL Instant Messenger, 893 Apache Software Foundation (ASF), Jakarta Struts, 60 Apache web server, 36 features, 37 mod_headers module, 812 module to handle compression, 814 typical HTTP header sent from, 811 APIs, web service, 618, 619–658 Ajax and, 657 blogging services, 620–623 bookmark services, 623–626 financial services, 626 mapping services, 627–631 news and weather services, 636–641 NewsIsFree, 637–641 other services, 656 photo services, 641–649 Flickr, 642–649 reference, 892–915 reference services, 650 search services, 651 shopping services, 652–655 eBay, 653–655 appearance of an application (see visualization) appendChild( ) method, 108, 259, 260 Apple Mac OS X alert windows, 335 Safari web browser, 18 <applet> elements, id attribute (XHTML 1.0), 321 applets, 734 arcade games, 731 ArcWeb, 627, 893 array literals, 824 arrays, using to store changing information, 399 ASCII, 852 online list of character codes, 223 ASIN (Amazon Standard Item Number), 598 ASP (Active Server Pages), 39, 40 ASP.NET, 40 assemblies (.NET), 59 asynchronous, 864 Atom, 16, 614 support by browser engines, 18 attributes creating for DOM elements, 106 creating for DOM nodes, 107 <frame> elements, style attributes, 317 <frameset> element, 317 XHTML versus HTML form elements, 484 XML, 845 Audioscrobbler, 632 AWS (Amazon Web Services) gathering AWS response and formatting for client, 610 methods available with, 608 REST request, 609 SOAP request using, 598 SOAP request using PHP, 607 WSDL document (example), 601–604 B back and forward buttons (browsers), problems with Ajax, 245, 918 background check records, 667 backward compatibility, web developers and, 19 balance (application layout), 159 bar graph based on user-submitted data, 476–481 completed page with bar graph, 481 JavaScript handling response, request, and drawing, 478–480 page to request data for the graph, 476 server-side script handling dynamic bar graph request, 477 base64_encode( ) function, 312 BaseError object, 409 BBC, 637, 893 before pseudoclass (CSS), 297 behavior property, 314 Berlind, David, 661 Bible lookup service, 656 Big O notations (sorting algorithms), 268 binary transparency, 439 bindings element (WSDL), 601 Blinksale (invoice service), 626, 894 bloat, avoiding in web application design, 143 blogging services, 620–623 list of some popular blogging APIs, 620 using FeedBurner MgmtAPI in an application, 621–623 Blogmarks (bookmarking service), 624, 894 blogs, 155 body component of a web page, 792 breaking into smaller pieces, 793–795 book information database (ISBNdb), 650 bookmark services, 623–626 del.icio.us API, 624–626 listing of some popular APIs, 623 bookmarks, 244 problems with Ajax, 917 breadcrumbs, 221–226 creating with XHTML lists and CSS, 222 CSS styling rules, 222 dynamically creating, 225 importNode( ) function for Internet Explorer, 224 supplied by server response, 223 Bresenham line algorithm, 748, 750 Bresenham, Jack E., 750 broken links, testing in web applications, 26 browsers, 17–19 alert windows, 335 alpha transparency support, 439 application navigation and, 151 back and forward buttons, problems with Ajax, 245 bookmarks, problems with Ajax, 244 code for browser detection, 198–199 compressed content, 814 CSS specifications, 13 customizations, 363–367 character encoding, 366 font sizes, 366 stylesheets, 364–366 enhancing to become true applications, xiii font sizes, relative, 386 functioning of file menu example, 209 Gecko layout engine, 18 KHTML/WebCore layout engines, 18 other layout engines, 19 Index | 927 browsers (continued) page loading status, 240 plug-ins, 733 Flash, 733 Java applets, 734 Shockwave, 734 pop-up windows and, 362 Presto layout engine, 19 right mouse button clicks, 764 standards compliance and backward compatibility, 20 standards supported by browser engines, 17 tables populating, innerHTML or DOM methods, 259 problems with, 248 testing performance in web application development, 26 Trident layout engine, 18 web standards and, 10 XMLHttpRequest object implementations, 69 XPath implementations, 83 XSLT and XPath support, 17 bubble sorts, 268 business logic layer (BLL), 804 business records, 667 businesses advantages of Ajax applications, 672–674 ease of installation, 674 reducing costs, 673 communication through combined applications, 720 communication through file sharing, 691–703 file notification, 696–698 receiving the file, 698–703 sending a file, 692–696 communication through whiteboards, 703–719 building the board, 703–707 enhancing with pen color choices, 715–718 enhancing with stamps and shapes, 719 replicating rendering on all users’ screens, 707–714 real-time communication, 674–691 chat client, 678–686 chat server, 686–690 client/server communication, 675 connecting to chat, 675–678 928 | Index button_onclick function (example), 465 buttons image built into a CSS button, 184 navigation bar, 180–184 representing 3D, 183 buySAFE, 894 BVS Performance Center for Banks, 145 C C# .NET, adding compression to a site, 832 C# documentation comments, 26 Cache-Control header (HTTP), 813 Cairo graphics engine (Gecko), 19 CakePHP framework, 62 callbacks Ajax response, 864 Ajax.Request( ) method, 416 supporting Ajax calls to the server for animations, 481 capital letters, use in application text, 166 Cascading Style Sheets (see CSS) Casciano, Chris, 332 Castlevania (arcade game), 731 catch clause, 414 Cavedog’s Total Annihilation game, 724 cells collection, 136 CGI (Common Gateway Interface), 37 FastCGI, 37 Channel Definition Format (CDF), 15 character class (example) changes in direction, 770 collision bounding constraints, 754 collision detection and other functionality, 756–759 modified animation object as basis, 735–740 modified for different characters and animation sequences, 740–742 mouse click event handling, 749–753 movement functionality added, 743–747 starts and stops movement, event handling, 770 character encodings, 851 choosing in browsers, 366 character references, 851 character.js file, 778–783 chat application (example), 672–691 chat client, 678–686 adding events to input controls, 680 event-handling functions, SendMessage( ) and QuitChat( ), 681 JavaScript code to run Ajax chat client, 683–686 monitoring message queue on server and displaying new messages, 682 PHP that creates structure, 679 tracking current users, 683 chat server, 686–690 get_messages.php file, 688 get_users.php file, 690 logout.php file, 686 put_message.php file, 687 quote_smart( ) function to prevent SQL injection, 687 client/server communication, 675 connecting to chat, 675–678 working Ajax chat application, 690 chat services, 155 checkboxes (form controls), 493 custom, 499 properties, 499 setting error indicator to, 553 using id attributes to get values, 494 Chevron, Jemima, 718 chunks-of-data structuring, 236 circular collision detection, 759–762 Pythagorean theorem, 759 testing a point and a circle, 761 testing for a circle and a rectangle, 761 testing two circles for collision, 761 class attribute, using for quick customizations, 398 classes FCL (Framework Class Library), 58 specification in web application development, 25 client frameworks, 94–97 Dojo Toolkit, 94 DWR, 96 jQuery, 96 other, 97 Prototype, 95 objects used with Ajax, 99–102 Sarissa, 97 client request for a web service, 606 client side of Ajax applications modular coding, 791–803 CSS, 795–802 JavaScript, 802 XHTML, 791–795 optimizations, 818–830 JavaScript, 822–830 XHTML and CSS, 819–822 client/server architecture, 28 communication between client and server, 838 communication model for chat program, 675 data validation duties, 536 RPC for distributed computing, 595 web sites in the year 2000, 6 client-side errors, 409 notifying the user, 419 closePopUp( ) function, 340 CLR (Common Language Runtime), 40 CMSs (Content Management Systems), Zope and, 62 CNET, 653, 895 code examples in this book, xviii code optimization, 839 coincident lines, 764 collisions, 753–764 circular collision detection, 759–762 detection techniques, 753 handling, 772 linear collision detection, 762–764 rectangular collision detection, 754–759 color color themes switcher, creating, 392–397 contrasts between text and its background, 166 GIF image format, 256-color-palette limit, 437 palette-based GIF images, 436 true-color images in GIF file format, 438 color wheel, 718 colors.css file, 369 example of contents, 370 Colossal Cave Adventure, 726 comb sorts, 268 comma-separated value (CSV) files, 47 comments documenting code, 27 removing from CSS and XHTML files, 819 removing from JavaScript files, 822 XML, 849 commercial Ajax web applications, 32 Common Gateway Interface (see CGI) Common Intermediate Language (CIL), 58 Common Language Infrastructure (CLI), 59 Common Language Specification (CLS), 58 Common Type System (CTS), 58 communication functionality, 155 Index | 929 communication needs for business combining applications, 720 file sharing, 691–703 file notification, 696–698 receiving the file, 698–703 sending a file, 692–696 real-time communication, 674–691 chat client, 678–686 chat server, 686–690 client/server communication, 675 connecting to chat, 675–678 whiteboards, 703–719 building the board, 703–707 enhancing with pen color choices, 715–718 enhancing with stamps and shapes, 719 replicating rendering on all users’ screens, 707–714 compiled languages, 37 compression, 830–833 adding to a site using C# .NET, 832 adding to a site using PHP, 831 HTTP, 813–815 CompuServe, GIF format, 435 conditional catch clause, 414 confirmation window, 349–351 consistency importance in application layout, 161 importance in web application design, 150 content changes, Ajax and, 923 Control.Slider object, 389–392 controller (MVC model), 30 controls and widgets, design of, 316 cookies, 375–381 incorporating into style switcher object, 378–381 simple cookie object (example), 376–378 storing user customization information, 407 cost savings from using web applications, 673 Craigslist, 660 CREATE TABLE statement (SQL), 49 createDocumentFragment( ) method, 107 createElement( ) method, 106, 260 createTextNode( ) method, 107, 259, 260 createXMLHttpRequest( ) function, 71 credit cards, validating, 541–543 credit checks, 626 930 | Index “Cross-Browser Scripting with importNode( )”, 329 cross-browser compatibility, testing for web applications, 26 Crowther, William, 726 Crysis (FPS game), 723 CSS (Cascading Style Sheets), 5, 117–129 CSS1 specification, 13 CSS2 specification, 13 properties and JavaScript equivalents, 119–124 CSS2.1 specification, 13 CSS3 specification, 13 Dojo Toolkit drag-and-drop functionality, 457 drop-down menu solution, 188–191 font size switching, 387–389 forms, 498 image rollovers in browsers without scripting, 186 in tabs, 216 layout using, 250–252 modifying and removing style, 118 stylesheet manipulation methods, 118 modularity of CSS files, 795–802 media types, 800 style properties, 796–800 notification of errors in form data, 552–555 error rules, 553–554 rule switching with JavaScript, 555 numbering system for document and table of contents, 296 optimizations, 819–822 removing comments from files, 819 removing unnecessary whitespace, 819 shortening class and id names, 821 using shorthand notation, 822 pen color changes in whiteboard application, 716 PNG image to be used in animation, 441 Results objects, Google’s AJAX Search API, 592 rule types, 117 sample tabs using XHTML lists, 213–215 simple navigation bars styled with, 176–180 slide show application, Internet Explorer version, 314 slide show styling, 305–307 sortable list styling, 300 standards supported by browser engines, 18 style information, 126 style switching, structure for CSS files, 368–371 styles for a Windows-like file menu, 195–198 styling <div> element to replace <iframe>, 324 styling alert windows, 338 styling breadcrumbs, 222 Internet Explorer and, 223 styling links at bottom of a page, 226 support by ASP.NET, 40 tab content sections, hiding from view, 218 table styling, keeping with sorts, 280–283 Zen Garden site, 160, 332 separating structure from presentation, 333–334 CSS2Properties object, 119 cssRules collection, 128 CTS (Common Type System), 58 CURL package, 655 Custom Search Engine, 587 CustomError object (example), 423–426 modified to garner user feedback, 428–432 D Daily CSS Fun web site, 332 Dalton Mailing Service, Inc., 146 data access layer (DAL), 804 data access module (model in MVC), 30 data exchange formats, choosing between XML and JSON, 92–94 data sources (for mashups), 665–668 open source services, 668 public data, 665–667 background check records, 667 business records, 667 people searches, 667 public records, 666 selecting, 670 data validation, 534–562 Ajax client/server validation, 558–562 advantages of, 562 checking form fields on the fly, 558–561 CSS notification of errors, 552–555 CSS error rules, 553–554 rule switching with JavaScript, 555 feeds, 616 importance of, 534 server-side, 555–558 checking whether expected data was received, 556 protecting the database, 557 returning problems to the client, 557 value checks, 557 using JavaScript, 536–552 checking phone numbers, 539 Dojo Toolkit, 549–552 regular expressions, 538 validation object, 543–549 value checking, 537 data, Ajax optimization, 839 databases, 44–48 basic three-tier application design pattern, 29 errors, 412 IBM DB2, 45 indexing, 569, 836 interaction with, using frameworks, 64–67 ISBNdb, 650 logging errors to, 420 Microsoft SQL Server, 45 nonrelational database models, 47 open source, MySQL and PostgreSQL, 46 optimization of SQL, 808 Oracle, 45 protecting from attacks, 557 querying, steps involved, 65 saving form data sent from clients, 528 server-side code to store element position, 405 storing information for a draggable object, 404 storing user customization information, 407 storing whiteboard coordinates, 710 DataUnison eBay Research, 653, 895 Dave.TV (video service), 632, 895 dBASE, 47 declarations XML, 844 XSLT, 856 decode( ) method (json class), 89 decrementing operators, 829 del.icio.us (bookmark service), 623, 624–626, 896 adding a post programmatically using PHP, 624–626 parameters to add a post, 624 Index | 931 DELETE statement (SQL), 53 delivery company, mashup pinpointing truck locations, 671 demographic information service, 650 density (in application layout), 160 Department of Corrections, data on felonies, 667 descriptors, GIF, 435 design of Ajax interfaces (see interfaces, designing) design patterns, 28–31, 148 client/server, 28 MVC (model-view-controller), 30 three-tier (basic), 29 Unique URLs, 244 design phase Ajax web application development, 25 software development life cycle, 23 DHTML Tree (Zapatec), 234 Diablo game series, 729 dimensional databases, 47 <div> elements hints for user searches, 577 innerHTML property, placing content into windows, 347 making draggable, 455 popupContainer, 340 using with Ajax to replace <iframe> elements, 323–329 inserting content, 325–329 styling the <div> element, 324 whiteboard, 703 Django framework, 61 document fragments, 828 Document object getElementById( ) method, 494 loading XML string into, 81 loadXML( ) method, 79 methods used to create nodes in a document tree, 107 setProperty( ) method, 83 Document Object Model (see DOM) document tree, 103 root element or root node, 104 text element or text node, 104 W3C node types, 105 document.getElementsByClassName( ) function, 98 documentation, 26 932 | Index documentFragment object, constructing tables, 260 DocumentFragment objects, 107 appending to list of child nodes, 108 Dojo Toolkit, 94, 880–884 dojo.io.bind, 881 drag-and-drop functionality, 457–460 adding a handle to draggable object, 458 creating draggables and droppables, 458 methods for HtmlDragSource object, 459 script to enable, 457 effects, 468 online demo, 470 handling results, 881 JSON with dot notation, 882 moving objects, 464 sending form data, 883 sortable list drag-and-drop functionality, 301–302 validation objects, 549–552 widgets for building form elements and forms, 515–517 dollar sign function, $( ), 98 DOM (Document Object Model), 5, 103–140 changing styles, 117–129 Internet Explorer and, 127 style information, 126 creating elements, attributes, and objects, 106–108 Document object traversal, 105 DOM Level 1 specification, 13 DOM Level 2 specification, 13, 106 Internet Explorer and, 224 DOM Level 3 specification, 13 XPath support by browsers, 83 element, attribute, and object information, 112–115 methods, listed, 113 properties, listed, 114 events, 129–135 creating, 130 information about, 131–133 initializing, 130 Internet Explorer and, 133 Form object, 490 importNode( ) method, 327 innerHTML property and, 138–140 JavaScript optimization and, 827–828 keyboard and mouse events, 764 loading XML file into Document object, 79 methods for dynamic table creation, 258 modifying and removing elements, attributes, and objects, 108–112 methods, listed, 110–112 parsing, 77 problems with using tables for layout, 248 specifications, 13 standardized list of DOM node types, 104 standards supported by browser engines, 18 tables, 135–138 traversing the DOM document tree, 115–117 methods, listed, 116 traversal properties, listed, 116 dot notation, 882 dragdrop components, 802 Draggable object, 344–345, 403, 455–456 change callback function, 456 container with handle for dragging point, 456 optional parameters, 455 Draggable object (Rico), 460 Draggables object, 455 dragging and dropping, 157 animation technique, 454–464 Dojo Toolkit, 457–460 Rico library, 460 script.aculo.us objects, 455–457 wz_dragdrop.js library, 460–463 Zapatec library, 463 sortable lists, 297–302 DrawCanvasUpdate( ) function, 713 drawing libraries, 472–481 JavaScript Vector Graphics Library, 472–481 Ajax application, 476–481 methods available to, 474–476 DrawPoint( ) method, 706 driving directions searches MapQuest classic site, 6 MapQuest site using Ajax, 8 drop downs (form controls), 493 custom, 505–515 properties, 505 preparing for the addition of Ajax, 495 drop-down menus, 188–191 Droppables object, 456 add( ) method, optional parameters, 456 callback functions, 457 Dropzone object (Rico), 460 DTDs (document type definitions), 853 Frameset, 316 SOAP documents and, 598 Dun and Bradstreet Credit Check, 626, 896 DWR framework, 96 dynamic content, providing with CGI, 37 with FastCGI, 37 with servlets, 38 with SSI, 38 dynamic directions (character movement), 747 E eBay, 653, 653–655, 896 API and documentation, 653 input parameters for REST requests, 654 requesting search results from REST API, 654 REST methods available in the API, 654 ECMA (European Computer Manufacturer’s Association) International, 12 Ecma International, 10 ECMAScript, 12 ActionScript implementation, 733 e-commerce sites, 652 economic games, 725 Edition 4 of ECMA-262, 19 Education and Outreach Working Group (WAI), 168 educational environment (web applications), 32 Edwards, Dean, 314 Effect object Accordion( ) method, 237 changing to open multiple sections, 238 Position( ), 466 SlideUp( ) and SlideDown( ) methods, 239 Effect object (Rico), 464 Index | 933 effects and dragdrop components, 802 effects on objects (animation), 467–472 Dojo Toolkit, 468 script.aculo.us, 467 Zapatec Effects library, 470–472 efficiency of web applications, 150 electronic products data center, 653 Element interface, 106 Element object hide( ) method, 242, 340 removeClassName( ) and add ClassName( ) methods, 283 show( ) method, 242, 340 elements DOM, 106 form, 482–484 SOAP document, 598 WSDL, 599–601 XML, 845 XSLT, 856–861 email addresses, validating, 540 embedded interpreters, FastCGI and, 37 empty( ) method, 556 encode( ) method, 90 Encoding namespace (SOAP), 598 end users, expecting too much of, 147 English Standard Version (ESV) Bible Lookup, 656, 897 Enterprise Resource Planning (ERP) packages, 62 entity headers (HTTP), 810 entity references, 849 Envelope namespace (SOAP), 598 environments (Ajax web applications), 31–33 commercial, 32 educational, 32 government, 33 intranet, 31 specific content, 33 equality testing, 537 ERP5, 62 error levels, 421 defining custom error levels, 422 errors, 408–433 displaying user errors, 420–433 following site design, 421–433 handling, 417–420 emailing the developer, 420 logging to a database, 420 notifying the user, 418 ignorable, 417 JavaScript, 409 934 | Index requiring immediate attention, 417 server-side, 410–413 database, 412 external errors, 413 server scripting, 410–412 trapping, 414–417 Ajax requests, 416 throwing an error, 415 try...catch...finally block, 414 escaping potentially dangerous characters to the database, 557 ESPN main page, 143 ESV (English Standard Version) Bible Lookup, 656, 897 Ethernet protocol, 816 packets, 817 European Computer Manufacturer’s Association (ECMA) International, 12 EvalError object, 409 event handling, 767–776 changes in direction, 770 receiving data, 774 starts and stops, 767–770 user input, 767 event listeners, 131 Event object, 132 constants and methods, 342 observe( ) method, 343, 442 pointerX( ) and pointerY( ) methods, 704, 766 EventCapturer object, 132 EventListener object, 132 events DOM, 129–135 creating, 130 focus and, 343 information about, 131–133 initializing, 130 Internet Explorer and, 133 modules, 129 importNode( ) function that registers events, 327–329 replacement of DOM Events with XML Events, 11 Events object (example), 767–770 changes in direction, 770 collision events, 773 making requests and parsing commands from server, 774 events.js file, 783–786 Excel, 47 execution speed, 809 execution time, 808 optimizing in JavaScript, 826 Expires response header (HTTP), 813 Extensible HyperText Markup Language (see XHTML) Extensible Markup Language (see XML) Extensible Stylesheet Language Transformation (see XSLT) extension blocks (GIF), 436, 438 F Facebook, 656, 897 Faces.com, 642, 897 FastCGI, 37 FCL (Framework Class Library), 58 FedEx, 898 feed aggregators, 614 feedback in Ajax web application design, 152 getting from users regarding errors, 428–432 FeedBlitz (blogging service), 620, 898 FeedBurner web services, 620, 898 MgmtAPI, 621 FeedMap, 627, 898 Fielding, Roy, 605 <fieldset> elements, 483 grouping associated form elements, 489 setting error indicator to checkboxes and radio buttons, 553 file menu, 192–212 adding Ajax, 210 code for browser detection, 198–199 CSS styles for Windows-like file menu, 195–198 JavaScript code for manipulating, 199–209 XHTML file, 192–195 file sharing, 691–703 file notification, 696–698 checking server for file notices, 696 get_file_notices.php file, 697 JSON response from server, 697 receiving the file, 698–703 delete_file.php, 700 PHP file checking indicator and giving response to sender, 702 sending user client monitoring for receiving, 700–702 server handling of get_file.php request, 698 sending a file, 692–696 PHP file to create form for file transfer, 692–694 saving file and alerting receiving user, 694 file size, 807, 808 file uploads from client to server, 529 financial services, 626 Firefox character encoding changes, 367 feedback agent, 428 text size changes, 367 user changes to browser theme, 363 :first-child pseudoselector (CSS), 223 first-person shooter (FPS) games, 722 “Fixing the Back Button and Enabling Bookmarking for Ajax Apps”, 244 Flash, 733 flat file databases, 47 flexibility of web applications, 150 Flickr, 155, 641, 899 example REST request, 642 example REST response, 642 JSON response to client, 649 methods available within the API, 643–648 organizing tools, 155 response formats, 642 REST request with JSON response to client (example), 648 focal point (of an application), 161 focus keeping focus on a pop-up alert box, 339–343 poor focus in web application design, 144 focusOnPopUP( ) function (example), 340 fonts, 162–167 browser font size changes, 366 color contrasts, 166 commanding attention to text, 166 font families and their types, 163–166 spacing of text, 166 switching font sizes, 386–392 CSS font file, 387–389 font-size slider bar, 389–392 relative font sizes, 386 use of capital letters, 166 fonts.css file, 369 example of contents, 370 Food Candy, 656 for loops, optimization techniques, 826 form buttons, 493 Index | 935 <form> elements, 483 <fieldset> element, 489 id attribute (XHTML 1.0), 321 name attribute, 490 Form object properties, 493 submit( ) and reset( ) methods, 493 Form.Validation object (example), 543–549 added functionality for Ajax, 558–561 forms, 28, 154, 482–533 accessibility, 485–488 aligning controls using CSS, 251 changing appearance of, 498 using CSS, 498 creating custom form controls drop downs, 505–515 radio buttons and checkboxes, 499–505 custom form objects in libraries and toolkits, 515–519 Dojo Toolkit, 515–517 Zapatec library, 518–519 elements, 482–484 file transfer form, 692–694 GSearchForm object, 585 HTML forms, replacement with XForms, 11 larger form in the navigation window, 353–355 manipulation with JavaScript, 490–497 sending data with Dojo Toolkit, 883 server handling of Ajax request, 524–531 emailing form data sent from the client, 527 GET/POST/RAW POST requests, 525 getting file uploads, 529 saving form data in a database, 528 sending data back to the client, 529 server responses, 531–533 client handling of complex response, 532 reporting success or failure, 531 submission with MooTools, 880 submitting a form using Ajax, 519–524 code example, 522–524 function looping through elements and getting values, 521 switching to different languages, 399–400 (see also data validation) forums, 155 forward button (browsers), problems with Ajax, 918 936 | Index FPS (first-person shooter) games, 722 fragment identifiers creating unique URL for bookmarks, 244 setting unique URLs for browser back button, 245 <frame> elements, 317 id attribute (XHTML 1.0), 321 frames, 316–323 animation, 442 animations with PNG format, 439 approximating iframes using Ajax and a <div> element, 323–329 inserting content, 325–329 styling the <div> element, 324 <frameset> and <frame> elements, 317 frameset, complete (example), 318 iframe, 319 packets, 816 XHTML and, 321–323 deprecation of frames and iframes, 321 if frames must be used, 322 using iframes as frames, 322 <frameset> elements, 317 Framework Class Library (FCL), 58 frameworks, 57 advantages of, 63–67 database interaction, 64 database interaction using Zend Framework, 65 client, 94–97 Dojo toolkit, 94 DWR, 96 jQuery, 96 other, 97 Prototype, 95 Sarissa, 97 Java, 60 Jakarta Struts, 60 Spring, 60 Tapestry, 61 moving objects, 464 .NET Framework, 58 PHP, 62 CakePHP, 62 Zend, 63 Zoop, 63 Python, 61 Ruby on Rails, 59 Sarissa, 83 using for animation, 453 (see also listings under names of individual frameworks) full text site searches, 568 functionality, 153–158 code components based on, 802 common web tools, 153–155 determining tools needed, 157 porting desktop functionality to the Web, 153 tool tips, 355 tools in desktop applications, 156 types of functions in applications, 153 functions (XSLT), 861 G games, 721–786 character animation, 735–753 moving the character, 742–753 walking loop, 735–742 collisions, 753–764 circular collision detection, 759–762 linear collision detection, 762–764 rectangular collision detection, 754–759 complete character.js file, 778–783 complete events.js file, 783–786 complete logic.js file, 776–778 event handling, 767–776 changes in direction, 770 collisions, 772 receiving data, 774 starts and stops, 767–770 user input, 767 Internet requirements, 732–735 game development with Ajax, 734 plug-ins for browsers, 733 resources for further information, 786 user input, 764–766 keyboard input, 765 mouse input, 766 web game genres, 721 adventure games, 726–727 arcade games, 731 first-person shooters (FPS), 722 other games, 732 puzzle games, 730 role-playing games (RPGs), 728–730 strategy games, 724–726 Gecko future of, 19 standards supported, 18 general headers (HTTP), 810 generating data, 157 GeoRSS, 630 Gestalt effect, 183 GET and POST methods, 72, 525 parameters passed with Prototype Ajax object, 865 submitting form data with Ajax, 520 getAttribute( ) method, 113 getDomDocument( ) method, 86 getElementById( ) method, 77, 105, 494 getElementsByTagName( ) method, 77, 105 GIF image format creating true-color GIF images, 438 history of, 435 how it works, 435–437 file structure, 435 palettes, 436 limitations of, 437 alpha transparency, 438 color depth, 437 global responders (Prototype), 867 global variables, 826 Gmail, 171 God-like strategy games, 725 golden ratio, 163 GoldenNumber.Net, 163 Goodman, Danny, 258 Google Advanced Search page, 576 AJAX Search API, 581–593, 652 displaying results, 589–593 GSearchControl object, 582–585 GSearchForm object, 585 GWebSearch object, 586 Searchers available with, 581 using the API, 587–589 Class Reference site, 587 Gmail service, 172 Google Maps, 7, 627 Google Reader, 614 implementing a site search tool using Google API, 570–575 map technology, use in HouseMaps.com mashup, 660 search capabilities, 565 search engine, 651 search engine used for a site, 581 sign up site for an account, 575 web service APIs, reference listing, 899–901 government agencies, information from, 651 Index | 937 government environments (web applications), 33 graphing charts from user data (see bar graph based on user-submitted data) graphs and charts, 157 GSearchControl object, 582–585 GSearchForm object, 585 Guha, R.

Send email to index@oreilly.com. 925 Ajax web applications (see RIAs) Ajax.NET, 97 Ajax.PeriodicalUpdater() object, 690, 868 Ajax.PeriodicUpdater object, 101 Ajax.Request object, 99 Ajax.Request( ) method, callbacks, 416 Ajax.Responders object, 867 Ajax.Updater object, 101, 867 options, 101 ajaxload.info web site, 436 Akismet (blogging API), 620, 892 alert boxes, 335 integrating the window into web applications, 335–347 keeping focus and closing the pop up, 339–343 moving the window, 344–347 window style, 336–339 variations caused by operating systems and browsers, 335 alpha channel, 438 alpha transparency, 438 support by PNG format, 440 alternate stylesheets, 364, 371 font size, 389 “Alternative Style: Working with Alternate Style Sheets”, 372 Amazon Standard Item Number (ASIN), 598 Amazon Web Services (see AWS) Amazon.com Book page, 154 breadcrumbs on, 221 organizing tools, 155 web services, 652 XML Scratch Pad, 609 Amnesty International, 656, 892 Amp’s main page, 144 animation, 434–481 Ajax, 453–481 dragging and dropping, 454–464 drawing libraries, 472–481 dynamically animating position of an object, 464–467 effects on objects, 467–472 object manipulations, 467–472 using frameworks, 453 building with PNG format, 439–453 CSS, 441 differences from GIFs, 440 JavaScript looping, 442–444 926 | Index more robust animation object, 444–448 using Ajax, 448–453 character animation for games, 735–753 moving the character, 742–753 walking loop, 735–742 GIF format history of, 435 how it works, 435–437 on the Web, 434 animation object (example), 442–444 adding Ajax, 448–453 starting, pausing, and stopping programmatically, 444–448 AOL Instant Messenger, 893 Apache Software Foundation (ASF), Jakarta Struts, 60 Apache web server, 36 features, 37 mod_headers module, 812 module to handle compression, 814 typical HTTP header sent from, 811 APIs, web service, 618, 619–658 Ajax and, 657 blogging services, 620–623 bookmark services, 623–626 financial services, 626 mapping services, 627–631 news and weather services, 636–641 NewsIsFree, 637–641 other services, 656 photo services, 641–649 Flickr, 642–649 reference, 892–915 reference services, 650 search services, 651 shopping services, 652–655 eBay, 653–655 appearance of an application (see visualization) appendChild( ) method, 108, 259, 260 Apple Mac OS X alert windows, 335 Safari web browser, 18 <applet> elements, id attribute (XHTML 1.0), 321 applets, 734 arcade games, 731 ArcWeb, 627, 893 array literals, 824 arrays, using to store changing information, 399 ASCII, 852 online list of character codes, 223 ASIN (Amazon Standard Item Number), 598 ASP (Active Server Pages), 39, 40 ASP.NET, 40 assemblies (.NET), 59 asynchronous, 864 Atom, 16, 614 support by browser engines, 18 attributes creating for DOM elements, 106 creating for DOM nodes, 107 <frame> elements, style attributes, 317 <frameset> element, 317 XHTML versus HTML form elements, 484 XML, 845 Audioscrobbler, 632 AWS (Amazon Web Services) gathering AWS response and formatting for client, 610 methods available with, 608 REST request, 609 SOAP request using, 598 SOAP request using PHP, 607 WSDL document (example), 601–604 B back and forward buttons (browsers), problems with Ajax, 245, 918 background check records, 667 backward compatibility, web developers and, 19 balance (application layout), 159 bar graph based on user-submitted data, 476–481 completed page with bar graph, 481 JavaScript handling response, request, and drawing, 478–480 page to request data for the graph, 476 server-side script handling dynamic bar graph request, 477 base64_encode( ) function, 312 BaseError object, 409 BBC, 637, 893 before pseudoclass (CSS), 297 behavior property, 314 Berlind, David, 661 Bible lookup service, 656 Big O notations (sorting algorithms), 268 binary transparency, 439 bindings element (WSDL), 601 Blinksale (invoice service), 626, 894 bloat, avoiding in web application design, 143 blogging services, 620–623 list of some popular blogging APIs, 620 using FeedBurner MgmtAPI in an application, 621–623 Blogmarks (bookmarking service), 624, 894 blogs, 155 body component of a web page, 792 breaking into smaller pieces, 793–795 book information database (ISBNdb), 650 bookmark services, 623–626 del.icio.us API, 624–626 listing of some popular APIs, 623 bookmarks, 244 problems with Ajax, 917 breadcrumbs, 221–226 creating with XHTML lists and CSS, 222 CSS styling rules, 222 dynamically creating, 225 importNode( ) function for Internet Explorer, 224 supplied by server response, 223 Bresenham line algorithm, 748, 750 Bresenham, Jack E., 750 broken links, testing in web applications, 26 browsers, 17–19 alert windows, 335 alpha transparency support, 439 application navigation and, 151 back and forward buttons, problems with Ajax, 245 bookmarks, problems with Ajax, 244 code for browser detection, 198–199 compressed content, 814 CSS specifications, 13 customizations, 363–367 character encoding, 366 font sizes, 366 stylesheets, 364–366 enhancing to become true applications, xiii font sizes, relative, 386 functioning of file menu example, 209 Gecko layout engine, 18 KHTML/WebCore layout engines, 18 other layout engines, 19 Index | 927 browsers (continued) page loading status, 240 plug-ins, 733 Flash, 733 Java applets, 734 Shockwave, 734 pop-up windows and, 362 Presto layout engine, 19 right mouse button clicks, 764 standards compliance and backward compatibility, 20 standards supported by browser engines, 17 tables populating, innerHTML or DOM methods, 259 problems with, 248 testing performance in web application development, 26 Trident layout engine, 18 web standards and, 10 XMLHttpRequest object implementations, 69 XPath implementations, 83 XSLT and XPath support, 17 bubble sorts, 268 business logic layer (BLL), 804 business records, 667 businesses advantages of Ajax applications, 672–674 ease of installation, 674 reducing costs, 673 communication through combined applications, 720 communication through file sharing, 691–703 file notification, 696–698 receiving the file, 698–703 sending a file, 692–696 communication through whiteboards, 703–719 building the board, 703–707 enhancing with pen color choices, 715–718 enhancing with stamps and shapes, 719 replicating rendering on all users’ screens, 707–714 real-time communication, 674–691 chat client, 678–686 chat server, 686–690 client/server communication, 675 connecting to chat, 675–678 928 | Index button_onclick function (example), 465 buttons image built into a CSS button, 184 navigation bar, 180–184 representing 3D, 183 buySAFE, 894 BVS Performance Center for Banks, 145 C C# .NET, adding compression to a site, 832 C# documentation comments, 26 Cache-Control header (HTTP), 813 Cairo graphics engine (Gecko), 19 CakePHP framework, 62 callbacks Ajax response, 864 Ajax.Request( ) method, 416 supporting Ajax calls to the server for animations, 481 capital letters, use in application text, 166 Cascading Style Sheets (see CSS) Casciano, Chris, 332 Castlevania (arcade game), 731 catch clause, 414 Cavedog’s Total Annihilation game, 724 cells collection, 136 CGI (Common Gateway Interface), 37 FastCGI, 37 Channel Definition Format (CDF), 15 character class (example) changes in direction, 770 collision bounding constraints, 754 collision detection and other functionality, 756–759 modified animation object as basis, 735–740 modified for different characters and animation sequences, 740–742 mouse click event handling, 749–753 movement functionality added, 743–747 starts and stops movement, event handling, 770 character encodings, 851 choosing in browsers, 366 character references, 851 character.js file, 778–783 chat application (example), 672–691 chat client, 678–686 adding events to input controls, 680 event-handling functions, SendMessage( ) and QuitChat( ), 681 JavaScript code to run Ajax chat client, 683–686 monitoring message queue on server and displaying new messages, 682 PHP that creates structure, 679 tracking current users, 683 chat server, 686–690 get_messages.php file, 688 get_users.php file, 690 logout.php file, 686 put_message.php file, 687 quote_smart( ) function to prevent SQL injection, 687 client/server communication, 675 connecting to chat, 675–678 working Ajax chat application, 690 chat services, 155 checkboxes (form controls), 493 custom, 499 properties, 499 setting error indicator to, 553 using id attributes to get values, 494 Chevron, Jemima, 718 chunks-of-data structuring, 236 circular collision detection, 759–762 Pythagorean theorem, 759 testing a point and a circle, 761 testing for a circle and a rectangle, 761 testing two circles for collision, 761 class attribute, using for quick customizations, 398 classes FCL (Framework Class Library), 58 specification in web application development, 25 client frameworks, 94–97 Dojo Toolkit, 94 DWR, 96 jQuery, 96 other, 97 Prototype, 95 objects used with Ajax, 99–102 Sarissa, 97 client request for a web service, 606 client side of Ajax applications modular coding, 791–803 CSS, 795–802 JavaScript, 802 XHTML, 791–795 optimizations, 818–830 JavaScript, 822–830 XHTML and CSS, 819–822 client/server architecture, 28 communication between client and server, 838 communication model for chat program, 675 data validation duties, 536 RPC for distributed computing, 595 web sites in the year 2000, 6 client-side errors, 409 notifying the user, 419 closePopUp( ) function, 340 CLR (Common Language Runtime), 40 CMSs (Content Management Systems), Zope and, 62 CNET, 653, 895 code examples in this book, xviii code optimization, 839 coincident lines, 764 collisions, 753–764 circular collision detection, 759–762 detection techniques, 753 handling, 772 linear collision detection, 762–764 rectangular collision detection, 754–759 color color themes switcher, creating, 392–397 contrasts between text and its background, 166 GIF image format, 256-color-palette limit, 437 palette-based GIF images, 436 true-color images in GIF file format, 438 color wheel, 718 colors.css file, 369 example of contents, 370 Colossal Cave Adventure, 726 comb sorts, 268 comma-separated value (CSV) files, 47 comments documenting code, 27 removing from CSS and XHTML files, 819 removing from JavaScript files, 822 XML, 849 commercial Ajax web applications, 32 Common Gateway Interface (see CGI) Common Intermediate Language (CIL), 58 Common Language Infrastructure (CLI), 59 Common Language Specification (CLS), 58 Common Type System (CTS), 58 communication functionality, 155 Index | 929 communication needs for business combining applications, 720 file sharing, 691–703 file notification, 696–698 receiving the file, 698–703 sending a file, 692–696 real-time communication, 674–691 chat client, 678–686 chat server, 686–690 client/server communication, 675 connecting to chat, 675–678 whiteboards, 703–719 building the board, 703–707 enhancing with pen color choices, 715–718 enhancing with stamps and shapes, 719 replicating rendering on all users’ screens, 707–714 compiled languages, 37 compression, 830–833 adding to a site using C# .NET, 832 adding to a site using PHP, 831 HTTP, 813–815 CompuServe, GIF format, 435 conditional catch clause, 414 confirmation window, 349–351 consistency importance in application layout, 161 importance in web application design, 150 content changes, Ajax and, 923 Control.Slider object, 389–392 controller (MVC model), 30 controls and widgets, design of, 316 cookies, 375–381 incorporating into style switcher object, 378–381 simple cookie object (example), 376–378 storing user customization information, 407 cost savings from using web applications, 673 Craigslist, 660 CREATE TABLE statement (SQL), 49 createDocumentFragment( ) method, 107 createElement( ) method, 106, 260 createTextNode( ) method, 107, 259, 260 createXMLHttpRequest( ) function, 71 credit cards, validating, 541–543 credit checks, 626 930 | Index “Cross-Browser Scripting with importNode( )”, 329 cross-browser compatibility, testing for web applications, 26 Crowther, William, 726 Crysis (FPS game), 723 CSS (Cascading Style Sheets), 5, 117–129 CSS1 specification, 13 CSS2 specification, 13 properties and JavaScript equivalents, 119–124 CSS2.1 specification, 13 CSS3 specification, 13 Dojo Toolkit drag-and-drop functionality, 457 drop-down menu solution, 188–191 font size switching, 387–389 forms, 498 image rollovers in browsers without scripting, 186 in tabs, 216 layout using, 250–252 modifying and removing style, 118 stylesheet manipulation methods, 118 modularity of CSS files, 795–802 media types, 800 style properties, 796–800 notification of errors in form data, 552–555 error rules, 553–554 rule switching with JavaScript, 555 numbering system for document and table of contents, 296 optimizations, 819–822 removing comments from files, 819 removing unnecessary whitespace, 819 shortening class and id names, 821 using shorthand notation, 822 pen color changes in whiteboard application, 716 PNG image to be used in animation, 441 Results objects, Google’s AJAX Search API, 592 rule types, 117 sample tabs using XHTML lists, 213–215 simple navigation bars styled with, 176–180 slide show application, Internet Explorer version, 314 slide show styling, 305–307 sortable list styling, 300 standards supported by browser engines, 18 style information, 126 style switching, structure for CSS files, 368–371 styles for a Windows-like file menu, 195–198 styling <div> element to replace <iframe>, 324 styling alert windows, 338 styling breadcrumbs, 222 Internet Explorer and, 223 styling links at bottom of a page, 226 support by ASP.NET, 40 tab content sections, hiding from view, 218 table styling, keeping with sorts, 280–283 Zen Garden site, 160, 332 separating structure from presentation, 333–334 CSS2Properties object, 119 cssRules collection, 128 CTS (Common Type System), 58 CURL package, 655 Custom Search Engine, 587 CustomError object (example), 423–426 modified to garner user feedback, 428–432 D Daily CSS Fun web site, 332 Dalton Mailing Service, Inc., 146 data access layer (DAL), 804 data access module (model in MVC), 30 data exchange formats, choosing between XML and JSON, 92–94 data sources (for mashups), 665–668 open source services, 668 public data, 665–667 background check records, 667 business records, 667 people searches, 667 public records, 666 selecting, 670 data validation, 534–562 Ajax client/server validation, 558–562 advantages of, 562 checking form fields on the fly, 558–561 CSS notification of errors, 552–555 CSS error rules, 553–554 rule switching with JavaScript, 555 feeds, 616 importance of, 534 server-side, 555–558 checking whether expected data was received, 556 protecting the database, 557 returning problems to the client, 557 value checks, 557 using JavaScript, 536–552 checking phone numbers, 539 Dojo Toolkit, 549–552 regular expressions, 538 validation object, 543–549 value checking, 537 data, Ajax optimization, 839 databases, 44–48 basic three-tier application design pattern, 29 errors, 412 IBM DB2, 45 indexing, 569, 836 interaction with, using frameworks, 64–67 ISBNdb, 650 logging errors to, 420 Microsoft SQL Server, 45 nonrelational database models, 47 open source, MySQL and PostgreSQL, 46 optimization of SQL, 808 Oracle, 45 protecting from attacks, 557 querying, steps involved, 65 saving form data sent from clients, 528 server-side code to store element position, 405 storing information for a draggable object, 404 storing user customization information, 407 storing whiteboard coordinates, 710 DataUnison eBay Research, 653, 895 Dave.TV (video service), 632, 895 dBASE, 47 declarations XML, 844 XSLT, 856 decode( ) method (json class), 89 decrementing operators, 829 del.icio.us (bookmark service), 623, 624–626, 896 adding a post programmatically using PHP, 624–626 parameters to add a post, 624 Index | 931 DELETE statement (SQL), 53 delivery company, mashup pinpointing truck locations, 671 demographic information service, 650 density (in application layout), 160 Department of Corrections, data on felonies, 667 descriptors, GIF, 435 design of Ajax interfaces (see interfaces, designing) design patterns, 28–31, 148 client/server, 28 MVC (model-view-controller), 30 three-tier (basic), 29 Unique URLs, 244 design phase Ajax web application development, 25 software development life cycle, 23 DHTML Tree (Zapatec), 234 Diablo game series, 729 dimensional databases, 47 <div> elements hints for user searches, 577 innerHTML property, placing content into windows, 347 making draggable, 455 popupContainer, 340 using with Ajax to replace <iframe> elements, 323–329 inserting content, 325–329 styling the <div> element, 324 whiteboard, 703 Django framework, 61 document fragments, 828 Document object getElementById( ) method, 494 loading XML string into, 81 loadXML( ) method, 79 methods used to create nodes in a document tree, 107 setProperty( ) method, 83 Document Object Model (see DOM) document tree, 103 root element or root node, 104 text element or text node, 104 W3C node types, 105 document.getElementsByClassName( ) function, 98 documentation, 26 932 | Index documentFragment object, constructing tables, 260 DocumentFragment objects, 107 appending to list of child nodes, 108 Dojo Toolkit, 94, 880–884 dojo.io.bind, 881 drag-and-drop functionality, 457–460 adding a handle to draggable object, 458 creating draggables and droppables, 458 methods for HtmlDragSource object, 459 script to enable, 457 effects, 468 online demo, 470 handling results, 881 JSON with dot notation, 882 moving objects, 464 sending form data, 883 sortable list drag-and-drop functionality, 301–302 validation objects, 549–552 widgets for building form elements and forms, 515–517 dollar sign function, $( ), 98 DOM (Document Object Model), 5, 103–140 changing styles, 117–129 Internet Explorer and, 127 style information, 126 creating elements, attributes, and objects, 106–108 Document object traversal, 105 DOM Level 1 specification, 13 DOM Level 2 specification, 13, 106 Internet Explorer and, 224 DOM Level 3 specification, 13 XPath support by browsers, 83 element, attribute, and object information, 112–115 methods, listed, 113 properties, listed, 114 events, 129–135 creating, 130 information about, 131–133 initializing, 130 Internet Explorer and, 133 Form object, 490 importNode( ) method, 327 innerHTML property and, 138–140 JavaScript optimization and, 827–828 keyboard and mouse events, 764 loading XML file into Document object, 79 methods for dynamic table creation, 258 modifying and removing elements, attributes, and objects, 108–112 methods, listed, 110–112 parsing, 77 problems with using tables for layout, 248 specifications, 13 standardized list of DOM node types, 104 standards supported by browser engines, 18 tables, 135–138 traversing the DOM document tree, 115–117 methods, listed, 116 traversal properties, listed, 116 dot notation, 882 dragdrop components, 802 Draggable object, 344–345, 403, 455–456 change callback function, 456 container with handle for dragging point, 456 optional parameters, 455 Draggable object (Rico), 460 Draggables object, 455 dragging and dropping, 157 animation technique, 454–464 Dojo Toolkit, 457–460 Rico library, 460 script.aculo.us objects, 455–457 wz_dragdrop.js library, 460–463 Zapatec library, 463 sortable lists, 297–302 DrawCanvasUpdate( ) function, 713 drawing libraries, 472–481 JavaScript Vector Graphics Library, 472–481 Ajax application, 476–481 methods available to, 474–476 DrawPoint( ) method, 706 driving directions searches MapQuest classic site, 6 MapQuest site using Ajax, 8 drop downs (form controls), 493 custom, 505–515 properties, 505 preparing for the addition of Ajax, 495 drop-down menus, 188–191 Droppables object, 456 add( ) method, optional parameters, 456 callback functions, 457 Dropzone object (Rico), 460 DTDs (document type definitions), 853 Frameset, 316 SOAP documents and, 598 Dun and Bradstreet Credit Check, 626, 896 DWR framework, 96 dynamic content, providing with CGI, 37 with FastCGI, 37 with servlets, 38 with SSI, 38 dynamic directions (character movement), 747 E eBay, 653, 653–655, 896 API and documentation, 653 input parameters for REST requests, 654 requesting search results from REST API, 654 REST methods available in the API, 654 ECMA (European Computer Manufacturer’s Association) International, 12 Ecma International, 10 ECMAScript, 12 ActionScript implementation, 733 e-commerce sites, 652 economic games, 725 Edition 4 of ECMA-262, 19 Education and Outreach Working Group (WAI), 168 educational environment (web applications), 32 Edwards, Dean, 314 Effect object Accordion( ) method, 237 changing to open multiple sections, 238 Position( ), 466 SlideUp( ) and SlideDown( ) methods, 239 Effect object (Rico), 464 Index | 933 effects and dragdrop components, 802 effects on objects (animation), 467–472 Dojo Toolkit, 468 script.aculo.us, 467 Zapatec Effects library, 470–472 efficiency of web applications, 150 electronic products data center, 653 Element interface, 106 Element object hide( ) method, 242, 340 removeClassName( ) and add ClassName( ) methods, 283 show( ) method, 242, 340 elements DOM, 106 form, 482–484 SOAP document, 598 WSDL, 599–601 XML, 845 XSLT, 856–861 email addresses, validating, 540 embedded interpreters, FastCGI and, 37 empty( ) method, 556 encode( ) method, 90 Encoding namespace (SOAP), 598 end users, expecting too much of, 147 English Standard Version (ESV) Bible Lookup, 656, 897 Enterprise Resource Planning (ERP) packages, 62 entity headers (HTTP), 810 entity references, 849 Envelope namespace (SOAP), 598 environments (Ajax web applications), 31–33 commercial, 32 educational, 32 government, 33 intranet, 31 specific content, 33 equality testing, 537 ERP5, 62 error levels, 421 defining custom error levels, 422 errors, 408–433 displaying user errors, 420–433 following site design, 421–433 handling, 417–420 emailing the developer, 420 logging to a database, 420 notifying the user, 418 ignorable, 417 JavaScript, 409 934 | Index requiring immediate attention, 417 server-side, 410–413 database, 412 external errors, 413 server scripting, 410–412 trapping, 414–417 Ajax requests, 416 throwing an error, 415 try...catch...finally block, 414 escaping potentially dangerous characters to the database, 557 ESPN main page, 143 ESV (English Standard Version) Bible Lookup, 656, 897 Ethernet protocol, 816 packets, 817 European Computer Manufacturer’s Association (ECMA) International, 12 EvalError object, 409 event handling, 767–776 changes in direction, 770 receiving data, 774 starts and stops, 767–770 user input, 767 event listeners, 131 Event object, 132 constants and methods, 342 observe( ) method, 343, 442 pointerX( ) and pointerY( ) methods, 704, 766 EventCapturer object, 132 EventListener object, 132 events DOM, 129–135 creating, 130 focus and, 343 information about, 131–133 initializing, 130 Internet Explorer and, 133 modules, 129 importNode( ) function that registers events, 327–329 replacement of DOM Events with XML Events, 11 Events object (example), 767–770 changes in direction, 770 collision events, 773 making requests and parsing commands from server, 774 events.js file, 783–786 Excel, 47 execution speed, 809 execution time, 808 optimizing in JavaScript, 826 Expires response header (HTTP), 813 Extensible HyperText Markup Language (see XHTML) Extensible Markup Language (see XML) Extensible Stylesheet Language Transformation (see XSLT) extension blocks (GIF), 436, 438 F Facebook, 656, 897 Faces.com, 642, 897 FastCGI, 37 FCL (Framework Class Library), 58 FedEx, 898 feed aggregators, 614 feedback in Ajax web application design, 152 getting from users regarding errors, 428–432 FeedBlitz (blogging service), 620, 898 FeedBurner web services, 620, 898 MgmtAPI, 621 FeedMap, 627, 898 Fielding, Roy, 605 <fieldset> elements, 483 grouping associated form elements, 489 setting error indicator to checkboxes and radio buttons, 553 file menu, 192–212 adding Ajax, 210 code for browser detection, 198–199 CSS styles for Windows-like file menu, 195–198 JavaScript code for manipulating, 199–209 XHTML file, 192–195 file sharing, 691–703 file notification, 696–698 checking server for file notices, 696 get_file_notices.php file, 697 JSON response from server, 697 receiving the file, 698–703 delete_file.php, 700 PHP file checking indicator and giving response to sender, 702 sending user client monitoring for receiving, 700–702 server handling of get_file.php request, 698 sending a file, 692–696 PHP file to create form for file transfer, 692–694 saving file and alerting receiving user, 694 file size, 807, 808 file uploads from client to server, 529 financial services, 626 Firefox character encoding changes, 367 feedback agent, 428 text size changes, 367 user changes to browser theme, 363 :first-child pseudoselector (CSS), 223 first-person shooter (FPS) games, 722 “Fixing the Back Button and Enabling Bookmarking for Ajax Apps”, 244 Flash, 733 flat file databases, 47 flexibility of web applications, 150 Flickr, 155, 641, 899 example REST request, 642 example REST response, 642 JSON response to client, 649 methods available within the API, 643–648 organizing tools, 155 response formats, 642 REST request with JSON response to client (example), 648 focal point (of an application), 161 focus keeping focus on a pop-up alert box, 339–343 poor focus in web application design, 144 focusOnPopUP( ) function (example), 340 fonts, 162–167 browser font size changes, 366 color contrasts, 166 commanding attention to text, 166 font families and their types, 163–166 spacing of text, 166 switching font sizes, 386–392 CSS font file, 387–389 font-size slider bar, 389–392 relative font sizes, 386 use of capital letters, 166 fonts.css file, 369 example of contents, 370 Food Candy, 656 for loops, optimization techniques, 826 form buttons, 493 Index | 935 <form> elements, 483 <fieldset> element, 489 id attribute (XHTML 1.0), 321 name attribute, 490 Form object properties, 493 submit( ) and reset( ) methods, 493 Form.Validation object (example), 543–549 added functionality for Ajax, 558–561 forms, 28, 154, 482–533 accessibility, 485–488 aligning controls using CSS, 251 changing appearance of, 498 using CSS, 498 creating custom form controls drop downs, 505–515 radio buttons and checkboxes, 499–505 custom form objects in libraries and toolkits, 515–519 Dojo Toolkit, 515–517 Zapatec library, 518–519 elements, 482–484 file transfer form, 692–694 GSearchForm object, 585 HTML forms, replacement with XForms, 11 larger form in the navigation window, 353–355 manipulation with JavaScript, 490–497 sending data with Dojo Toolkit, 883 server handling of Ajax request, 524–531 emailing form data sent from the client, 527 GET/POST/RAW POST requests, 525 getting file uploads, 529 saving form data in a database, 528 sending data back to the client, 529 server responses, 531–533 client handling of complex response, 532 reporting success or failure, 531 submission with MooTools, 880 submitting a form using Ajax, 519–524 code example, 522–524 function looping through elements and getting values, 521 switching to different languages, 399–400 (see also data validation) forums, 155 forward button (browsers), problems with Ajax, 918 936 | Index FPS (first-person shooter) games, 722 fragment identifiers creating unique URL for bookmarks, 244 setting unique URLs for browser back button, 245 <frame> elements, 317 id attribute (XHTML 1.0), 321 frames, 316–323 animation, 442 animations with PNG format, 439 approximating iframes using Ajax and a <div> element, 323–329 inserting content, 325–329 styling the <div> element, 324 <frameset> and <frame> elements, 317 frameset, complete (example), 318 iframe, 319 packets, 816 XHTML and, 321–323 deprecation of frames and iframes, 321 if frames must be used, 322 using iframes as frames, 322 <frameset> elements, 317 Framework Class Library (FCL), 58 frameworks, 57 advantages of, 63–67 database interaction, 64 database interaction using Zend Framework, 65 client, 94–97 Dojo toolkit, 94 DWR, 96 jQuery, 96 other, 97 Prototype, 95 Sarissa, 97 Java, 60 Jakarta Struts, 60 Spring, 60 Tapestry, 61 moving objects, 464 .NET Framework, 58 PHP, 62 CakePHP, 62 Zend, 63 Zoop, 63 Python, 61 Ruby on Rails, 59 Sarissa, 83 using for animation, 453 (see also listings under names of individual frameworks) full text site searches, 568 functionality, 153–158 code components based on, 802 common web tools, 153–155 determining tools needed, 157 porting desktop functionality to the Web, 153 tool tips, 355 tools in desktop applications, 156 types of functions in applications, 153 functions (XSLT), 861 G games, 721–786 character animation, 735–753 moving the character, 742–753 walking loop, 735–742 collisions, 753–764 circular collision detection, 759–762 linear collision detection, 762–764 rectangular collision detection, 754–759 complete character.js file, 778–783 complete events.js file, 783–786 complete logic.js file, 776–778 event handling, 767–776 changes in direction, 770 collisions, 772 receiving data, 774 starts and stops, 767–770 user input, 767 Internet requirements, 732–735 game development with Ajax, 734 plug-ins for browsers, 733 resources for further information, 786 user input, 764–766 keyboard input, 765 mouse input, 766 web game genres, 721 adventure games, 726–727 arcade games, 731 first-person shooters (FPS), 722 other games, 732 puzzle games, 730 role-playing games (RPGs), 728–730 strategy games, 724–726 Gecko future of, 19 standards supported, 18 general headers (HTTP), 810 generating data, 157 GeoRSS, 630 Gestalt effect, 183 GET and POST methods, 72, 525 parameters passed with Prototype Ajax object, 865 submitting form data with Ajax, 520 getAttribute( ) method, 113 getDomDocument( ) method, 86 getElementById( ) method, 77, 105, 494 getElementsByTagName( ) method, 77, 105 GIF image format creating true-color GIF images, 438 history of, 435 how it works, 435–437 file structure, 435 palettes, 436 limitations of, 437 alpha transparency, 438 color depth, 437 global responders (Prototype), 867 global variables, 826 Gmail, 171 God-like strategy games, 725 golden ratio, 163 GoldenNumber.Net, 163 Goodman, Danny, 258 Google Advanced Search page, 576 AJAX Search API, 581–593, 652 displaying results, 589–593 GSearchControl object, 582–585 GSearchForm object, 585 GWebSearch object, 586 Searchers available with, 581 using the API, 587–589 Class Reference site, 587 Gmail service, 172 Google Maps, 7, 627 Google Reader, 614 implementing a site search tool using Google API, 570–575 map technology, use in HouseMaps.com mashup, 660 search capabilities, 565 search engine, 651 search engine used for a site, 581 sign up site for an account, 575 web service APIs, reference listing, 899–901 government agencies, information from, 651 Index | 937 government environments (web applications), 33 graphing charts from user data (see bar graph based on user-submitted data) graphs and charts, 157 GSearchControl object, 582–585 GSearchForm object, 585 Guha, R.


pages: 292 words: 66,588

Learning Vue.js 2: Learn How to Build Amazing and Complex Reactive Web Applications Easily With Vue.js by Olga Filipova

Amazon Web Services, continuous integration, create, read, update, delete, en.wikipedia.org, Firefox, Google Chrome, MVC pattern, pull request, side project, single page application, Skype, source of truth, web application

I can leave my laptop on for the whole night, but I can't leave it on forever. Once I switch it off, the network connection is lost and there is no access to my application anymore. Also, even if I could leave it on forever, I don't like this website address. It's a bunch of letters and numbers, and I want it to be something meaningful. There are more robust ways. I can buy, for example, a virtual instance on AWS (Amazon Web Services), copy my application to this instance, buy a domain at a domain provider such as GoDaddy, associate this domain to the bought instance's IP, and run the application there and it will be accessible, maintained, backed up, and taken care of by the Amaz(on)ing service. Amazing, but ... expensive as hell. Let's think of this solution when our applications reach the corresponding size and payback level.


pages: 190 words: 62,941

Wild Ride: Inside Uber's Quest for World Domination by Adam Lashinsky

"side hustle", Airbnb, always be closing, Amazon Web Services, autonomous vehicles, Ayatollah Khomeini, business process, Chuck Templeton: OpenTable:, cognitive dissonance, corporate governance, DARPA: Urban Challenge, Donald Trump, Elon Musk, gig economy, Golden Gate Park, Google X / Alphabet X, information retrieval, Jeff Bezos, Lyft, Marc Andreessen, Mark Zuckerberg, megacity, Menlo Park, new economy, pattern recognition, price mechanism, ride hailing / ride sharing, Sand Hill Road, self-driving car, Silicon Valley, Silicon Valley startup, Skype, Snapchat, South of Market, San Francisco, sovereign wealth fund, statistical model, Steve Jobs, TaskRabbit, Tony Hsieh, transportation-network company, Travis Kalanick, turn-by-turn navigation, Uber and Lyft, Uber for X, uber lyft, ubercab, young professional

What won Sullivan over was the opportunity to oversee “real-world threats and digital threats,” to report directly to Kalanick, and to have “relatively unlimited resources” to accomplish his task. His mandate, he says, was to make safety a “brand differentiator” for Uber. Sullivan entered Uber in the spring of 2015 with a long to-do list. Securing driver and rider data was a matter of adding more rigorous authentication requirements to Uber’s account with Amazon Web Services, where Uber stored its data. “We went from basics to best practices,” he says. His team researched “telematics,” the ability to collect data over networks that measured movement. As an example, Uber’s data on how its drivers handle their phones is so precise it can tell if they are holding the phone while driving, a no-no. The company also can tell if drivers are braking too hard or going too fast.


The Data Journalism Handbook by Jonathan Gray, Lucy Chambers, Liliana Bounegru

Amazon Web Services, barriers to entry, bioinformatics, business intelligence, carbon footprint, citizen journalism, correlation does not imply causation, crowdsourcing, David Heinemeier Hansson, eurozone crisis, Firefox, Florence Nightingale: pie chart, game design, Google Earth, Hans Rosling, information asymmetry, Internet Archive, John Snow's cholera map, Julian Assange, linked data, moral hazard, MVC pattern, New Journalism, openstreetmap, Ronald Reagan, Ruby on Rails, Silicon Valley, social graph, SPARQL, text mining, web application, WikiLeaks

Things like creating and fetching items from the database, and matching URLs to specific code in an app are built into the frameworks, so developers don’t need to write code to do basic things like that. While there hasn’t been a formal survey of news app teams in the U.S., it is generally understood that most teams use one of these two frameworks for database-backed news apps. At ProPublica, we use Ruby on Rails. The development of rapid web server “slice” provisioning services like Amazon Web Services also took away some of what used to make deploying a web app a slow process. Apart from that, we use pretty standard tools to work with data: Google Refine and Microsoft Excel to clean data; SPSS and R to do statistics; ArcGIS and QGIS to do GIS; Git for source code management; TextMate, Vim and Sublime Text for writing code; and a mix of MySQL, PostgreSQL and SQL Server for databases. We built our own JavaScript framework called “Glass” that helps us build front-end heavy apps in JavaScript very quickly


pages: 270 words: 64,235

Effective Programming: More Than Writing Code by Jeff Atwood

AltaVista, Amazon Web Services, barriers to entry, cloud computing, endowment effect, Firefox, future of work, game design, Google Chrome, gravity well, job satisfaction, Khan Academy, Kickstarter, loss aversion, Marc Andreessen, Mark Zuckerberg, Merlin Mann, Minecraft, Paul Buchheit, Paul Graham, price anchoring, race to the bottom, recommendation engine, science of happiness, Skype, social software, Steve Jobs, web application, Y Combinator, zero-sum game

Software developers should share the customer’s pain. I know it’s not glamorous. But until you’ve demonstrated a willingness to help the customers using the software you’ve built — and more importantly, learn why they need help — you haven’t truly finished building that software. Working With the Chaos Monkey Late last year, the Netflix Tech Blog wrote about five lessons they learned moving to Amazon Web Services. AWS is, of course, the preeminent provider of so-called “cloud computing,” so this can essentially be read as key advice for any website considering a move to the cloud. And it’s great advice, too. Here’s the one bit that struck me as most essential: We’ve sometimes referred to the Netflix software architecture in AWS as our Rambo Architecture. Each system has to be able to succeed, no matter what, even all on its own.


Realtime Web Apps: HTML5 WebSocket, Pusher, and the Web’s Next Big Thing by Jason Lengstorf, Phil Leggetter

Amazon Web Services, barriers to entry, don't repeat yourself, en.wikipedia.org, Firefox, Google Chrome, MVC pattern, Ruby on Rails, Skype, software as a service, web application, WebSocket

Unfortunately, to use one of these apps, users were originally required to enter their Twitter username and password to grant the app access to the account. This access was unrestricted, so users simply trusted that these app developers would be responsible and hoping for the best. Obviously, this was not a sustainable model. OAuth emerged as an alternative authentication protocol after the team behind it studied many of the proprietary solutions that existed—services such as Google AuthSub, AOL OpenAuth, and the Amazon Web Services API—and combined the best practices into an open protocol that would be easy for any service to use and any developer to implement. OAuth is currently working on the OAuth 2.0 draft, which has been implemented by several service providers, including Facebook. 1 http://oauth.net/ 259 Appendix a ■ OAuth How OAuth Works Before we talk about what’s happening, let’s look at a real-world OAuth workflow: social photo-sharing site Flickr is part of the Yahoo!


pages: 244 words: 66,977

Subscribed: Why the Subscription Model Will Be Your Company's Future - and What to Do About It by Tien Tzuo, Gabe Weisert

3D printing, Airbnb, airport security, Amazon Web Services, augmented reality, autonomous vehicles, blockchain, Build a better mousetrap, business cycle, business intelligence, business process, call centre, cloud computing, cognitive dissonance, connected car, death of newspapers, digital twin, double entry bookkeeping, Elon Musk, factory automation, fiat currency, Internet of things, inventory management, iterative process, Jeff Bezos, Kevin Kelly, Lean Startup, Lyft, manufacturing employment, minimum viable product, natural language processing, Network effects, Nicholas Carr, nuclear winter, pets.com, profit maximization, race to the bottom, ride hailing / ride sharing, Sand Hill Road, shareholder value, Silicon Valley, skunkworks, smart meter, social graph, software as a service, spice trade, Steve Ballmer, Steve Jobs, subscription business, Tim Cook: Apple, transport as a service, Uber and Lyft, uber lyft, Y2K, Zipcar

Those that fail to at least explore consumption-based offerings may end up on the path to obsolescence.” Across the board, perpetual license and maintenance revenues are slowing or in decline. There’s no growth left in on-premise software, and a lot of younger SaaS companies founded over the last ten years are starting to see real gains in market share. Hardware is shifting as well—the success of Amazon Web Services has convinced IT buyers to shift from big, expensive capex-based installations to opex-based rental agreements. The big firms are struggling to catch up because they can see where the market is heading. Today’s innovative companies are increasingly pursuing recurring revenue–based business models and are relegating their ERP systems to a commodity general ledger in the process. The old, cumbersome “one size fits all” ERP model increasingly means that everything gets billed to Oracle, and nothing gets done particularly well.


pages: 237 words: 64,411

Humans Need Not Apply: A Guide to Wealth and Work in the Age of Artificial Intelligence by Jerry Kaplan

Affordable Care Act / Obamacare, Amazon Web Services, asset allocation, autonomous vehicles, bank run, bitcoin, Bob Noyce, Brian Krebs, business cycle, buy low sell high, Capital in the Twenty-First Century by Thomas Piketty, combinatorial explosion, computer vision, corporate governance, crowdsourcing, en.wikipedia.org, Erik Brynjolfsson, estate planning, Flash crash, Gini coefficient, Goldman Sachs: Vampire Squid, haute couture, hiring and firing, income inequality, index card, industrial robot, information asymmetry, invention of agriculture, Jaron Lanier, Jeff Bezos, job automation, John Markoff, John Maynard Keynes: Economic Possibilities for our Grandchildren, Loebner Prize, Mark Zuckerberg, mortgage debt, natural language processing, Own Your Own Home, pattern recognition, Satoshi Nakamoto, school choice, Schrödinger's Cat, Second Machine Age, self-driving car, sentiment analysis, Silicon Valley, Silicon Valley startup, Skype, software as a service, The Chicago School, The Future of Employment, Turing test, Watson beat the top human players on Jeopardy!, winner-take-all economy, women in the workforce, working poor, Works Progress Administration

John Markoff, “Researchers Announce Advance in Image-Recognition Software,” New York Times, November 17, 2014, science section. 5. “Strawberry Harvesting Robot,” posted by meminsider, YouTube, November 30, 2010, http://youtu.be/uef6ayK8ilY. 6. For an amazingly insightful analysis of the effects of increased communication and decreased energy cost across everything from living cells to civilizations, see Robert Wright, Nonzero (New York: Pantheon 2000). 7. Amazon Web Services (AWS), accessed November 25, 2014, http://aws.amazon.com. 8. W. B. Yeats, “The Second Coming,” 1919, http://en.wikipedia.org/wiki/The_Second_Coming_(poem). 3. ROBOTIC PICKPOCKETS 1. At least, that’s the way I remember it. Dave may have a different recollection, especially in light of the fact that Raiders wasn’t released until 1981. 2. David Elliot Shaw, “Evolution of the NON-VON Supercomputer,” Columbia University Computer Science Technical Reports, 1983, http://hdl.handle.net/10022/AC:P:11591. 3. http://en.wikipedia.org/wiki/MapReduce, last modified December 31, 2014. 4.


pages: 602 words: 177,874

Thank You for Being Late: An Optimist's Guide to Thriving in the Age of Accelerations by Thomas L. Friedman

3D printing, additive manufacturing, affirmative action, Airbnb, AltaVista, Amazon Web Services, autonomous vehicles, Ayatollah Khomeini, barriers to entry, Berlin Wall, Bernie Sanders, bitcoin, blockchain, Bob Noyce, business cycle, business process, call centre, centre right, Chris Wanstrath, Clayton Christensen, clean water, cloud computing, corporate social responsibility, creative destruction, crowdsourcing, David Brooks, demand response, demographic dividend, demographic transition, Deng Xiaoping, Donald Trump, Erik Brynjolfsson, failed state, Fall of the Berlin Wall, Ferguson, Missouri, first square of the chessboard / second half of the chessboard, Flash crash, game design, gig economy, global pandemic, global supply chain, illegal immigration, immigration reform, income inequality, indoor plumbing, intangible asset, Intergovernmental Panel on Climate Change (IPCC), Internet of things, invention of the steam engine, inventory management, Irwin Jacobs: Qualcomm, Jeff Bezos, job automation, John Markoff, John von Neumann, Khan Academy, Kickstarter, knowledge economy, knowledge worker, land tenure, linear programming, Live Aid, low skilled workers, Lyft, Marc Andreessen, Mark Zuckerberg, mass immigration, Maui Hawaii, Menlo Park, Mikhail Gorbachev, mutually assured destruction, Nelson Mandela, pattern recognition, planetary scale, pull request, Ralph Waldo Emerson, ransomware, Ray Kurzweil, Richard Florida, ride hailing / ride sharing, Robert Gordon, Ronald Reagan, Second Machine Age, self-driving car, shareholder value, sharing economy, Silicon Valley, Skype, smart cities, South China Sea, Steve Jobs, supercomputer in your pocket, TaskRabbit, The Rise and Fall of American Growth, Thomas L Friedman, transaction costs, Transnistria, uber lyft, undersea cable, urban decay, urban planning, Watson beat the top human players on Jeopardy!, WikiLeaks, women in the workforce, Y2K, Yogi Berra, zero-sum game

Indeed, these digital flows have become so rich and powerful they are to the twenty-first century what rivers running off mountains were to civilization and cities in days of old. Back then, you wanted to build your town or your factory along a rushing river—such as the Amazon—and let it flow through you. That river would give you power, mobility, nourishment, and access to neighbors and their ideas. So it is with these digital flows into and out of the supernova. But the rivers you want to build on now are Amazon Web Services or Microsoft’s Azure—giant connectors that enable you, your business, or your nation to get access to all the computing-power applications in the supernova, where you can tie into every flow in the world in which you want to participate. The world cannot get this connected in so many new realms at such profound new depths without being reshaped. And this chapter is about how these digital global flows are doing just that: enabling so many more people around the world to access the supernova’s technology toolbox to become makers and breakers; making the world so much more interdependent in financial terms, so every country is now more vulnerable to every other country’s economy; driving contact between strangers at a pace and scale we’ve never seen before, so that good and bad ideas can go viral and extinguish and manufacture prejudices much more quickly; making every leader more exposed and transparent; and ensuring that the price countries pay for adventures abroad will be much higher than they expect, making these flows a new source of geopolitical restraint.

Louis Park Agadez, Niger age of accelerations; dislocation and; education and; human adaptability as challenged by; as inflection point; innovation as response to; leadership and; the Machine and; Moore’s law and; social technologies and agriculture: in Africa and Middle East; climate change and; monocultures vs. polycultures in Airbnb; trust and air-conditioning Aita, Samir algorithms; human oversight and; self-improving Alivio Capital Allen, Paul Allisam, Graham Almaniq, Mati Al Qaeda Al-Shabab AltaVista Amazon (company) Amazon rain forest Amazon Web Services American Civil Liberties Union American Dream American Interest American University of Iraq “America’s New Immigrant Entrepreneurs: Then and Now” (Kauffman Foundation) Amman, Jordan amplifying, as geopolitical policy Andersen, Jeanne Anderson, Chris Anderson, Ross Anderson, Wendell Andreessen, Marc Andrews, Garrett Android AngularJS Annan, Kofi Anthropocene epoch Anthropocene Review anti-Semitism APIs (application programming interfaces) Apple; see also Jobs, Steve Applebaum, Anne Apple Newton Apple Pay apps revolution Arab Awakening Arabic, author’s study of Arab-Muslim world, golden age of Arafat, Yasser architects, software for Armstrong, Neil artificial intelligence (AI); intelligent algorithms and; intelligent assistance and Artnet.com Ashe, Neil Ashraf, Quamrul Assad, Bashar al- Associated Press Astren, Fred AT&T; intelligent assistance and; iPhone gamble of; lifelong learning and; as software company Atkinson, Karen atmosphere: aerosol loading in; CO2 in; ozone layer of ATMs Auguste, Byron Austria Austro-Hungarian Empire Autodesk automation, see computers, computing autonomous systems; see also cars, self-driving Autor, David Avaaz.org Azmar Mountain Bajpai, Aloke Baker, James A., III balance of power Bandar Mahshahr, Iran bandwidth Bangladesh bankruptcy laws bank tellers Barbut, Monique baseball, class-mixing and BASIC Bass, Carl Batman, Turkey BBCNews.com Bee, Samantha Beinhocker, Eric Beirut: civil war in; 1982 Israeli-Palestinian war in Bell, Alexander Graham Bell Labs Bennis, Warren Benyus, Janine Berenberg, Morrie Berenberg, Tess Berkus, Nate Berlin, Isaiah Berlin Wall, fall of Bessen, James Betsiboka River “Better Outcomes Through Radical Inclusion” (Wells) Between Debt and the Devil (Turner) Beykpour, Kayvon Bible Bigbelly garbage cans big data; consumers and; financial services and; software innovation and; supernova and Big Shift Big World, Small Planet (Rockström) “Big Yellow Taxi” (song) Bingham, Marjorie bin Laden, Osama bin Yehia, Abdullah biodiversity: environmental niches and; resilience and biodiversity loss; climate change and biofuels biogeochemical flows biomass fuels biotechnology bioweapons birth control, opposition to Bitcoin black elephants Blase, Bill blockchain technology Bloomberg.com Blumenfeld, Isadore “Kid Cann” Bobby Z (Bobby Rivkin) Bodin, Wes Bohr, Mark Bojia, Ayele Z.


pages: 274 words: 75,846

The Filter Bubble: What the Internet Is Hiding From You by Eli Pariser

A Declaration of the Independence of Cyberspace, A Pattern Language, Amazon Web Services, augmented reality, back-to-the-land, Black Swan, borderless world, Build a better mousetrap, Cass Sunstein, citizen journalism, cloud computing, cognitive dissonance, crowdsourcing, Danny Hillis, data acquisition, disintermediation, don't be evil, Filter Bubble, Flash crash, fundamental attribution error, global village, Haight Ashbury, Internet of things, Isaac Newton, Jaron Lanier, Jeff Bezos, jimmy wales, Kevin Kelly, knowledge worker, Mark Zuckerberg, Marshall McLuhan, megacity, Metcalfe’s law, Netflix Prize, new economy, PageRank, paypal mafia, Peter Thiel, recommendation engine, RFID, Robert Metcalfe, sentiment analysis, shareholder value, Silicon Valley, Silicon Valley startup, social graph, social software, social web, speech recognition, Startup school, statistical model, stem cell, Steve Jobs, Steven Levy, Stewart Brand, technoutopianism, the scientific method, urban planning, Whole Earth Catalog, WikiLeaks, Y Combinator

Rather than housing their Web sites and databases internally, many businesses and start-ups now run on virtual computers in vast server farms managed by other companies. The enormous pool of computing power and storage these networked machines create is known as the cloud, and it allows clients much greater flexibility. If your business runs in the cloud, you don’t need to buy more hardware when your processing demands expand: You just rent a greater portion of the cloud. Amazon Web Services, one of the major players in the space, hosts thousands of Web sites and Web servers and undoubtedly stores the personal data of millions. On one hand, the cloud gives every kid in his or her basement access to nearly unlimited computing power to quickly scale up a new online service. On the other, as Clive Thompson pointed out to me, the cloud “is actually just a handful of companies.” When Amazon booted the activist Web site WikiLeaks off its servers under political pressure in 2010, the site immediately collapsed—there was nowhere to go.


pages: 260 words: 76,223

Ctrl Alt Delete: Reboot Your Business. Reboot Your Life. Your Future Depends on It. by Mitch Joel

3D printing, Amazon Web Services, augmented reality, call centre, clockwatching, cloud computing, Firefox, future of work, ghettoisation, Google Chrome, Google Glasses, Google Hangouts, Khan Academy, Kickstarter, Kodak vs Instagram, Lean Startup, Marc Andreessen, Mark Zuckerberg, Network effects, new economy, Occupy movement, place-making, prediction markets, pre–internet, QR code, recommendation engine, Richard Florida, risk tolerance, self-driving car, Silicon Valley, Silicon Valley startup, Skype, social graph, social web, Steve Jobs, Steve Wozniak, Thomas L Friedman, Tim Cook: Apple, Tony Hsieh, white picket fence, WikiLeaks, zero-sum game

Google continues to fascinate as the search engine expands into areas like online video (YouTube), mobile (Android and the Nexus line of devices), email services (Gmail), Web browsers (Google Chrome), online social networking (Google+), and beyond (self-driving cars and Google Glasses). Amazon continues to squiggle by pushing beyond selling books online into e-readers (Kindle), selling shoes (Zappos), offering cloud computing technology (Amazon Web Services), and beyond. When you actually start digging down deep into how these companies have evolved and stayed relevant, you won’t see business models that look like anything from the playbooks of Kodak or RIM. These organizations are in a constant state of rebooting with teams of people who are actively guiding their own careers as they squiggle. Even MySpace is making another run at it by being squiggly.


pages: 254 words: 76,064

Whiplash: How to Survive Our Faster Future by Joi Ito, Jeff Howe

3D printing, Albert Michelson, Amazon Web Services, artificial general intelligence, basic income, Bernie Sanders, bitcoin, Black Swan, blockchain, Burning Man, buy low sell high, Claude Shannon: information theory, cloud computing, Computer Numeric Control, conceptual framework, crowdsourcing, cryptocurrency, data acquisition, disruptive innovation, Donald Trump, double helix, Edward Snowden, Elon Musk, Ferguson, Missouri, fiat currency, financial innovation, Flash crash, frictionless, game design, Gerolamo Cardano, informal economy, interchangeable parts, Internet Archive, Internet of things, Isaac Newton, Jeff Bezos, John Harrison: Longitude, Joi Ito, Khan Academy, Kickstarter, Mark Zuckerberg, microbiome, Nate Silver, Network effects, neurotypical, Oculus Rift, pattern recognition, peer-to-peer, pirate software, pre–internet, prisoner's dilemma, Productivity paradox, race to the bottom, RAND corporation, random walk, Ray Kurzweil, Ronald Coase, Ross Ulbricht, Satoshi Nakamoto, self-driving car, SETI@home, side project, Silicon Valley, Silicon Valley startup, Simon Singh, Singularitarianism, Skype, slashdot, smart contracts, Steve Ballmer, Steve Jobs, Steven Levy, Stewart Brand, Stuxnet, supply-chain management, technological singularity, technoutopianism, The Nature of the Firm, the scientific method, The Signal and the Noise by Nate Silver, There's no reason for any individual to have a computer in his home - Ken Olsen, Thomas Kuhn: the structure of scientific revolutions, universal basic income, unpaid internship, uranium enrichment, urban planning, WikiLeaks

The developers often share the inner workings of the game and even allow fans to use copyrighted content to create videos or other derivative goods. It’s very hard to see where the company ends and the customer begins in these systems. You see pull at work not only with parts and labor, but with financial capital as well. Kickstarter allows people to raise what they want in a fashion that’s far more agile and responsive than traditional fund-raising methods. Crowdfunding demonstrates that the same logic behind Amazon Web Services—the “distributed computing” division—works for the aggregation of financial capital as well. People often associate crowdfunding with dubious ideas for new products, but Experiment.com shows that the same system can be used to fund serious scientific research.17 Beyond crowdfunding, crowdsourcing also provides independent creators with affordable options for extending their resources. Rather than hiring large teams of engineers, designers, and programmers, start-ups and individuals can tap into a global community of freelancers and volunteers who can provide the skills they lack.


Mastering Structured Data on the Semantic Web: From HTML5 Microdata to Linked Open Data by Leslie Sikos

AGPL, Amazon Web Services, bioinformatics, business process, cloud computing, create, read, update, delete, Debian, en.wikipedia.org, fault tolerance, Firefox, Google Chrome, Google Earth, information retrieval, Infrastructure as a Service, Internet of things, linked data, natural language processing, openstreetmap, optical character recognition, platform as a service, search engine result page, semantic web, Silicon Valley, social graph, software as a service, SPARQL, text mining, Watson beat the top human players on Jeopardy!, web application, wikimedia commons

For Linked Data access, there is an RDFizer service, 115 Chapter 4 ■ Semantic Web Development Tools a SPARQL Endpoint, and you can also define HTTP headers. The default host for RDFizer and the SPARQL endpoint is linkeddata.uriburner.com, which can be modified arbitrarily. The RDFizer is Virtuoso Sponger (http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main/VirtSponger), a component of Virtuoso’s SPARQL Processor and Proxy Web Service. Sponger supports RDFa, GRDDL, Amazon Web Services, eBay Web Services, Freebase Web Services, Facebook Web Services, Yahoo! Finance, XBRL Instance documents, Document Object Identifiers (DOI), RSS and Atom feeds, ID3 tags of MP3 music files, vCard, Microformats, Flickr, and Del.icio.us contents. OpenLink Data Explorer handles RDF, Turtle, and Notation3 MIME data. The default viewer for MIME data is Virtuoso Describe, but you can also choose Virtuoso About or Virtuoso ODE with or without SSL.


pages: 255 words: 78,207

Web Scraping With Python: Collecting Data From the Modern Web by Ryan Mitchell

AltaVista, Amazon Web Services, cloud computing, en.wikipedia.org, Firefox, Guido van Rossum, meta analysis, meta-analysis, natural language processing, optical character recognition, random walk, self-driving car, Turing test, web application

Google Compute Engine by Marc Cohen, Kathryn Hurley, and Paul Newson is a straightforward resource on using Google Cloud Computing with both Python and JavaScript. Not only does it cover Google’s user interface, but also the command-line and scripting tools that you can use to give your application greater flexibility. If you prefer to work with Amazon, Mitch Garnaat’s Python and AWS Cookbook is a brief but extremely useful guide that will get you started with Amazon Web Services and show you how to get a scalable application up and running. Moving Forward The Web is constantly changing. The technologies that bring us images, video, text, and other data files are constantly being updated and reinvented. To keep pace, the collection of technologies used to scrape data from the Internet must also change. Future versions of this text omit JavaScript entirely as an obsolete and rarely used technology and instead focus on HTML8 hologram parsing.


pages: 302 words: 73,581

Platform Scale: How an Emerging Business Model Helps Startups Build Large Empires With Minimum Investment by Sangeet Paul Choudary

3D printing, Airbnb, Amazon Web Services, barriers to entry, bitcoin, blockchain, business process, Chuck Templeton: OpenTable:, Clayton Christensen, collaborative economy, commoditize, crowdsourcing, cryptocurrency, data acquisition, frictionless, game design, hive mind, Internet of things, invisible hand, Kickstarter, Lean Startup, Lyft, M-Pesa, Marc Andreessen, Mark Zuckerberg, means of production, multi-sided market, Network effects, new economy, Paul Graham, recommendation engine, ride hailing / ride sharing, shareholder value, sharing economy, Silicon Valley, Skype, Snapchat, social graph, social software, software as a service, software is eating the world, Spread Networks laid a new fibre optics cable between New York and Chicago, TaskRabbit, the payments system, too big to fail, transport as a service, two-sided market, Uber and Lyft, Uber for X, uber lyft, Wave and Pay

Uber offers benefits and vehicle purchase schemes to drivers who want to buy a new car and start participating on Uber. THE RESOURCE BARRIER It is much easier to start a business in 2015 than it was in 1995. One of the most important reasons is the significant reduction in the amount of resources required to get a business up and running. An important contributor to this change is the rise of Amazon Web Services, which lowers the amount of resources required at the outset to start up. A startup that would have needed to procure a minimum level of infrastructure in 1995 may leverage Amazon’s resources on demand in 2015. THE ACCESS BARRIER Platforms often disrupt gatekeepers by giving producers direct access to potential consumers. Most media businesses (publishing, performing arts) are industries with gatekeepers that determine which producers get market access.


pages: 999 words: 194,942

Clojure Programming by Chas Emerick, Brian Carper, Christophe Grand

Amazon Web Services, Benoit Mandelbrot, cloud computing, continuous integration, database schema, domain-specific language, don't repeat yourself, en.wikipedia.org, failed state, finite state, Firefox, game design, general-purpose programming language, Guido van Rossum, Larry Wall, mandelbrot fractal, Paul Graham, platform as a service, premature optimization, random walk, Ruby on Rails, Schrödinger's Cat, semantic web, software as a service, sorting algorithm, Turing complete, type inference, web application

We’ll take a look at one, Amazon’s Elastic Beanstalk service, that is broadly applicable to Clojure web applications, automating the provisioning and configuration of servers and deployment of applications to those servers. Deploying Clojure Apps to Amazon’s Elastic Beanstalk Amazon’s Elastic Beanstalk (EB) is a platform as a service that provides a thin layer of automation and deployment management tools on top of Amazon Web Services’s (AWS) lower-level EC2 compute and load balancer services. EB allows you to programmatically provision and control environments (collections of one or more application servers fronted by a load balancer), to which you can deploy different versions of your application. The load balancers used by EB are integrated with this provisioning mechanism, so that when your application experiences higher load (based on metrics you define, such as number of requests or aggregate bandwidth utilized per minute), the corresponding EB environment is expanded to contain more app servers to service that load.

Clojure on Heroku While Chapter 17 presented a very typical web application deployment approach using a container (such as Tomcat on Amazon’s Elastic Beanstalk), there are ways to run Clojure web apps without a container, and therefore without the packaging work that containers imply. While such containerless deployment options are a relatively new approach in the JVM space, they are becoming more and more common. One of the most popular to date is provided by Heroku (http://heroku.com)—a scalable application deployment platform that is itself hosted on Amazon Web Services—which now directly supports the deployment of Ring-based web applications using Leiningen[450] without requiring any separate compilation or packaging steps. Heroku has the added benefit of offering application “add-ons”—managed database clusters, message queues, web services, and so on—that you can configure and use from within your Clojure project without having to set up and manage such things yourself.


pages: 270 words: 79,992

The End of Big: How the Internet Makes David the New Goliath by Nicco Mele

4chan, A Declaration of the Independence of Cyberspace, Airbnb, Amazon Web Services, Any sufficiently advanced technology is indistinguishable from magic, Apple's 1984 Super Bowl advert, barriers to entry, Berlin Wall, big-box store, bitcoin, business climate, call centre, Cass Sunstein, centralized clearinghouse, Chelsea Manning, citizen journalism, cloud computing, collaborative consumption, collaborative editing, commoditize, creative destruction, crony capitalism, cross-subsidies, crowdsourcing, David Brooks, death of newspapers, disruptive innovation, Donald Trump, Douglas Engelbart, Douglas Engelbart, en.wikipedia.org, Exxon Valdez, Fall of the Berlin Wall, Filter Bubble, Firefox, global supply chain, Google Chrome, Gordon Gekko, Hacker Ethic, Jaron Lanier, Jeff Bezos, jimmy wales, John Markoff, Julian Assange, Kevin Kelly, Khan Academy, Kickstarter, Lean Startup, Mark Zuckerberg, minimum viable product, Mitch Kapor, Mohammed Bouazizi, Mother of all demos, Narrative Science, new economy, Occupy movement, old-boy network, peer-to-peer, period drama, Peter Thiel, pirate software, publication bias, Robert Metcalfe, Ronald Reagan, Ronald Reagan: Tear down this wall, sharing economy, Silicon Valley, Skype, social web, Steve Jobs, Steve Wozniak, Stewart Brand, Stuxnet, Ted Nelson, Telecommunications Act of 1996, telemarketer, The Wisdom of Crowds, transaction costs, uranium enrichment, Whole Earth Catalog, WikiLeaks, Zipcar

Of course, when you’re surfing the Internet, you don’t care where the Web sites physically reside. You’re in the virtual cloud of the Internet, and the specific server—be it in Idaho, New York, or Shanghai—doesn’t affect your experience. Indeed, servers have become incredibly commoditized, with large volumes of computing power made available in seconds for pennies. Amazon has developed some notoriety in this area with a product called Amazon web services (AWS). In the process of building a giant infrastructure to sell everything, but especially books, over the Internet, Amazon realized that they could sell excess capacity on their server farms. Need a place to host your Web site? Buy it from Amazon. Suddenly need 10,000 times more space because your tiny start-up has gone viral? Amazon can turn it on in seconds. Hundreds if not thousands of vendors now offer cloud computing.


pages: 301 words: 85,126

AIQ: How People and Machines Are Smarter Together by Nick Polson, James Scott

Air France Flight 447, Albert Einstein, Amazon Web Services, Atul Gawande, autonomous vehicles, availability heuristic, basic income, Bayesian statistics, business cycle, Cepheid variable, Checklist Manifesto, cloud computing, combinatorial explosion, computer age, computer vision, Daniel Kahneman / Amos Tversky, Donald Trump, Douglas Hofstadter, Edward Charles Pickering, Elon Musk, epigenetics, Flash crash, Grace Hopper, Gödel, Escher, Bach, Harvard Computers: women astronomers, index fund, Isaac Newton, John von Neumann, late fees, low earth orbit, Lyft, Magellanic Cloud, mass incarceration, Moneyball by Michael Lewis explains big data, Moravec's paradox, more computing power than Apollo, natural language processing, Netflix Prize, North Sea oil, p-value, pattern recognition, Pierre-Simon Laplace, ransomware, recommendation engine, Ronald Reagan, self-driving car, sentiment analysis, side project, Silicon Valley, Skype, smart cities, speech recognition, statistical model, survivorship bias, the scientific method, Thomas Bayes, Uber for X, uber lyft, universal basic income, Watson beat the top human players on Jeopardy!, young professional

Any company with a new idea for making good use of its oil would have faced enormous fixed costs just to get started; as a result, most of the oil would have sat in the ground. Well, the same logic holds for data, the oil of the twenty-first century. Most hobbyists or small companies would face prohibitive costs if they had to buy all the gear and expertise needed to build an AI system from their data. But the cloud-computing resources provided by outfits like Microsoft Azure, IBM, and Amazon Web Services have turned that fixed cost into a variable cost, radically changing the economic calculus for large-scale data storage and analysis. Today, anyone who wants to make use of their “oil” can now do so cheaply, by renting someone else’s infrastructure. When you put those four trends together—faster chips, massive data sets, cloud computing, and above all good ideas—you get a supernova-like explosion in both the demand and capacity for using AI to solve real problems.


pages: 301 words: 89,076

The Globotics Upheaval: Globalisation, Robotics and the Future of Work by Richard Baldwin

agricultural Revolution, Airbnb, AltaVista, Amazon Web Services, augmented reality, autonomous vehicles, basic income, business process, business process outsourcing, call centre, Capital in the Twenty-First Century by Thomas Piketty, Cass Sunstein, commoditize, computer vision, Corn Laws, correlation does not imply causation, Credit Default Swap, David Ricardo: comparative advantage, declining real wages, deindustrialization, deskilling, Donald Trump, Douglas Hofstadter, Downton Abbey, Elon Musk, Erik Brynjolfsson, facts on the ground, future of journalism, future of work, George Gilder, Google Glasses, Google Hangouts, hiring and firing, impulse control, income inequality, industrial robot, intangible asset, Internet of things, invisible hand, James Watt: steam engine, Jeff Bezos, job automation, knowledge worker, laissez-faire capitalism, low skilled workers, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, manufacturing employment, Mark Zuckerberg, mass immigration, mass incarceration, Metcalfe’s law, new economy, optical character recognition, pattern recognition, Ponzi scheme, post-industrial society, post-work, profit motive, remote working, reshoring, ride hailing / ride sharing, Robert Gordon, Robert Metcalfe, Ronald Reagan, Second Machine Age, self-driving car, side project, Silicon Valley, Skype, Snapchat, social intelligence, sovereign wealth fund, standardized shipping container, statistical model, Stephen Hawking, Steve Jobs, supply-chain management, TaskRabbit, telepresence, telepresence robot, telerobotics, Thomas Malthus, trade liberalization, universal basic income

It is not perfect, but being able to Skype freely with someone who doesn’t speak your language is nothing short of marvelous. Microsoft and Amazon have entered the race as well. Microsoft is using its digital assistant, Cortana, to allow users to speak in any of twenty languages and have the results appear as text in up to sixty different languages. Its email app, Outlook, added an instant translation add-in in 2018. At the end of 2017, Amazon introduced its contender—Amazon Translate—via Amazon Web Services. Unbuilding the Tower of Babel The fact that machine translation is entering everyday life is a big change. As anyone who has traveled or done business internationally knows, language is a huge barrier to just about everything. There is even an Old Testament story that says language-linked divisiveness was divinely inspired. The passage, from the Book of Genesis, discusses a building that humans were constructing to reach the heavens: “The Lord said, ‘If as one people speaking the same language they have begun to do this, then nothing they plan to do will be impossible for them.


pages: 761 words: 80,914

Ansible: Up and Running: Automating Configuration Management and Deployment the Easy Way by Lorin Hochstein

Amazon Web Services, cloud computing, continuous integration, Debian, DevOps, domain-specific language, don't repeat yourself, general-purpose programming language, Infrastructure as a Service, job automation, MITM: man-in-the-middle, pull request, side project, smart transportation, web application

You can also assign IAM roles to running instances, so you can effectively say: “This instance is allowed to start other instances.” When you make requests against EC2 using a client program that supports IAM roles, and an instance is granted permissions by an IAM role, the client will fetch the credentials from the EC2 instance metadata service and use those to make requests against the EC2 service end point. You can create an IAM role through the Amazon Web Services (AWS) management console, or at the command line using the AWS Command Line Interface tool, or AWS CLI. AWS Management Console Here’s how you would use the AWS management console to create an IAM role that has “Power User Access,” meaning that it is permitted to do pretty much anything with AWS except to modify IAM users and groups. Log in to the AWS management console. Click on “Identity & Access Management.”


pages: 294 words: 81,292

Our Final Invention: Artificial Intelligence and the End of the Human Era by James Barrat

AI winter, AltaVista, Amazon Web Services, artificial general intelligence, Asilomar, Automated Insights, Bayesian statistics, Bernie Madoff, Bill Joy: nanobots, brain emulation, cellular automata, Chuck Templeton: OpenTable:, cloud computing, cognitive bias, commoditize, computer vision, cuban missile crisis, Daniel Kahneman / Amos Tversky, Danny Hillis, data acquisition, don't be evil, drone strike, Extropian, finite state, Flash crash, friendly AI, friendly fire, Google Glasses, Google X / Alphabet X, Isaac Newton, Jaron Lanier, John Markoff, John von Neumann, Kevin Kelly, Law of Accelerating Returns, life extension, Loebner Prize, lone genius, mutually assured destruction, natural language processing, Nicholas Carr, optical character recognition, PageRank, pattern recognition, Peter Thiel, prisoner's dilemma, Ray Kurzweil, Rodney Brooks, Search for Extraterrestrial Intelligence, self-driving car, semantic web, Silicon Valley, Singularitarianism, Skype, smart grid, speech recognition, statistical model, stealth mode startup, stem cell, Stephen Hawking, Steve Jobs, Steve Wozniak, strong AI, Stuxnet, superintelligent machines, technological singularity, The Coming Technological Singularity, Thomas Bayes, traveling salesman, Turing machine, Turing test, Vernor Vinge, Watson beat the top human players on Jeopardy!, zero day

In 2011, botnet victims increased 654 percent: Schwartz, Mathew, “Botnet Victims Increased 654 percent in 2011,” InformationWeek, February 18, 2011, http://www.informationweek.com/news/security/attacks/229218944?cid=RSSfeed_IWK_All (accessed July 11, 2012). a one trillion-dollar industry: Symantec, “What is Cybercrime?” last modified 2012, http://us.norton.com/cybercrime/definition.jsp (accessed July 11, 2012). Cloud computing has been a runaway success: Malik, Om, “How Big is Amazon’s Cloud Computing Business? Find Out,” GIGAOM, August 11, 2010, http://gigaom.com/cloud/amazon-web-services-revenues/ (accessed June 4, 2011). Zeus stole some $70 million: Ragan, Steve, “ZBot data dump discovered with over 74,000 FTP credentials,” The Tech Herald, June 29, 2009, http://www.thetechherald.com/articles/ZBot-data-dump-discovered-with-over-74-000-FTP-credentials/6514/ (accessed June 4, 2011). 21.3 percent overall, comes from Shaoxing: Melanson, Donald, “Symantec names Shaoxing, China, as world’s malware capital,” Engadget, March 29, 2010, http://www.engadget.com/2010/03/29/symantec-names-shaoxing-china-worlds-malware-capital (accessed June 4, 2011).


pages: 245 words: 83,272

Artificial Unintelligence: How Computers Misunderstand the World by Meredith Broussard

1960s counterculture, A Declaration of the Independence of Cyberspace, Ada Lovelace, AI winter, Airbnb, Amazon Web Services, autonomous vehicles, availability heuristic, barriers to entry, Bernie Sanders, bitcoin, Buckminster Fuller, Chris Urmson, Clayton Christensen, cloud computing, cognitive bias, complexity theory, computer vision, crowdsourcing, Danny Hillis, DARPA: Urban Challenge, digital map, disruptive innovation, Donald Trump, Douglas Engelbart, easy for humans, difficult for computers, Electric Kool-Aid Acid Test, Elon Musk, Firefox, gig economy, global supply chain, Google Glasses, Google X / Alphabet X, Hacker Ethic, Jaron Lanier, Jeff Bezos, John von Neumann, Joi Ito, Joseph-Marie Jacquard, life extension, Lyft, Mark Zuckerberg, mass incarceration, Minecraft, minimum viable product, Mother of all demos, move fast and break things, move fast and break things, Nate Silver, natural language processing, PageRank, payday loans, paypal mafia, performance metric, Peter Thiel, price discrimination, Ray Kurzweil, ride hailing / ride sharing, Ross Ulbricht, Saturday Night Live, school choice, self-driving car, Silicon Valley, speech recognition, statistical model, Steve Jobs, Steven Levy, Stewart Brand, Tesla Model S, the High Line, The Signal and the Noise by Nate Silver, theory of mind, Travis Kalanick, Turing test, Uber for X, uber lyft, Watson beat the top human players on Jeopardy!, Whole Earth Catalog, women in the workforce

We now have access to all of the functions in pandas and numpy. We can choose to import all of the functions or just a few. From scikit-learn, we’ll import only two functions. One is named tree and the other is named preprocessing. Next, let’s import the data from a comma-separated values (CSV) file that is also sitting somewhere on the Internet. Specifically, this CSV file is sitting on a server owned by Amazon Web Services (AWS). We can tell because the base URL of the file (the first part after http://) is s3.amazonaws.com. A CSV file is a file of structured data in which each column is separated by a comma. We’re going to import two different Titanic data files from AWS. One is a training data set, another is a test data set. Both data sets are in CSV format. Let’s import the data: train_url = "http://s3.amazonaws.com/assets.datacamp.com/course/Kaggle/train.csv" train = pd.read_csv(train_url) test_url = "http://s3.amazonaws.com/assets.datacamp.com/course/Kaggle/test.csv" test = pd.read_csv(test_url) pd.read_csv() means “please invoke the read_csv() function, which lives in the pd (pandas) library.”


pages: 299 words: 91,839

What Would Google Do? by Jeff Jarvis

23andMe, Amazon Mechanical Turk, Amazon Web Services, Anne Wojcicki, barriers to entry, Berlin Wall, business process, call centre, cashless society, citizen journalism, clean water, commoditize, connected car, credit crunch, crowdsourcing, death of newspapers, different worldview, disintermediation, diversified portfolio, don't be evil, fear of failure, Firefox, future of journalism, G4S, Google Earth, Googley, Howard Rheingold, informal economy, inventory management, Jeff Bezos, jimmy wales, Kevin Kelly, Mark Zuckerberg, moral hazard, Network effects, new economy, Nicholas Carr, old-boy network, PageRank, peer-to-peer lending, post scarcity, prediction markets, pre–internet, Ronald Coase, search inside the book, Silicon Valley, Skype, social graph, social software, social web, spectrum auction, speech recognition, Steve Jobs, the medium is the message, The Nature of the Firm, the payments system, The Wisdom of Crowds, transaction costs, web of trust, WikiLeaks, Y Combinator, Zipcar

He sells his retail services to other merchants, sending them customers online and taking a cut, in some cases warehousing and shipping their inventory and charging for the services. He also took the computer infrastructure he had to build and offered it to any company as a low-cost, pay-as-you-go service: computing power, storage, databases, and a mechanism for paying programmers. Countless companies now use Amazon Web Services as their backend, foregoing or at least forestalling investments in computers and software. Amazon has also created the infrastructure for an on-demand workforce called Mechanical Turk (named after a phony chess-playing automaton from 1769 that had a human chess master hidden inside). Companies post a repetitive task to be done and anyone can earn money—as little as one cent per task—by verifying the address in a picture, for example, or categorizing content.


pages: 713 words: 93,944

Seven Databases in Seven Weeks: A Guide to Modern Databases and the NoSQL Movement by Eric Redmond, Jim Wilson, Jim R. Wilson

AGPL, Amazon Web Services, create, read, update, delete, data is the new oil, database schema, Debian, domain-specific language, en.wikipedia.org, fault tolerance, full text search, general-purpose programming language, Kickstarter, linked data, MVC pattern, natural language processing, node package manager, random walk, recommendation engine, Ruby on Rails, Skype, social graph, web application

Using cURL allows us to peek at the underlying API without resorting to a particular driver or programming language. Riak is a great choice for datacenters like Amazon that must serve many requests with low latency. If every millisecond spent waiting is a potential customer loss, Riak is hard to beat. It’s easy to manage, easy to set up, and can grow with your needs. If you’ve ever used Amazon Web Services, like SimpleDB or S3, you may notice some similarities in form and function. This is no coincidence. Riak is inspired by Amazon’s Dynamo paper.[9] In this chapter, we’ll investigate how Riak stores and retrieves values and how to tie data together using Links. Then we’ll explore a data-retrieval concept used heavily throughout this book: mapreduce. We’ll see how Riak clusters its servers and handles requests, even in the face of server failure.


pages: 340 words: 96,149

@War: The Rise of the Military-Internet Complex by Shane Harris

Amazon Web Services, barriers to entry, Berlin Wall, Brian Krebs, centralized clearinghouse, clean water, computer age, crowdsourcing, data acquisition, don't be evil, Edward Snowden, failed state, Firefox, John Markoff, Julian Assange, mutually assured destruction, peer-to-peer, Silicon Valley, Silicon Valley startup, Skype, Stuxnet, undersea cable, uranium enrichment, WikiLeaks, zero day

It is, in effect, like the top-secret networks the military uses. It won’t be impervious to assault—neither are the military’s, as the Buckshot Yankee operation showed. But they will afford a higher level of security than what you have now in the mostly ungoverned expanse of the Internet. Who would build such a community? Perhaps Amazon. In fact, it has already built a version—for the CIA. Amazon Web Services, which hosts other companies’ data and computing operations, has a $600 million contract to build a private system, or cloud, for the spy agency. But unlike other clouds, which are accessed through the public Internet, this one will be run this one using Amazon’s own hardware and network equipment. Amazon hasn’t historically offered private clouds to its customers, but the CIA may be on the frontier of a new market.


pages: 285 words: 86,853

What Algorithms Want: Imagination in the Age of Computing by Ed Finn

Airbnb, Albert Einstein, algorithmic trading, Amazon Mechanical Turk, Amazon Web Services, bitcoin, blockchain, Chuck Templeton: OpenTable:, Claude Shannon: information theory, commoditize, Credit Default Swap, crowdsourcing, cryptocurrency, disruptive innovation, Donald Knuth, Douglas Engelbart, Douglas Engelbart, Elon Musk, factory automation, fiat currency, Filter Bubble, Flash crash, game design, Google Glasses, Google X / Alphabet X, High speed trading, hiring and firing, invisible hand, Isaac Newton, iterative process, Jaron Lanier, Jeff Bezos, job automation, John Conway, John Markoff, Just-in-time delivery, Kickstarter, late fees, lifelogging, Loebner Prize, Lyft, Mother of all demos, Nate Silver, natural language processing, Netflix Prize, new economy, Nicholas Carr, Norbert Wiener, PageRank, peer-to-peer, Peter Thiel, Ray Kurzweil, recommendation engine, Republic of Letters, ride hailing / ride sharing, Satoshi Nakamoto, self-driving car, sharing economy, Silicon Valley, Silicon Valley ideology, Silicon Valley startup, social graph, software studies, speech recognition, statistical model, Steve Jobs, Steven Levy, Stewart Brand, supply-chain management, TaskRabbit, technological singularity, technoutopianism, The Coming Technological Singularity, the scientific method, The Signal and the Noise by Nate Silver, The Structural Transformation of the Public Sphere, The Wealth of Nations by Adam Smith, transaction costs, traveling salesman, Turing machine, Turing test, Uber and Lyft, Uber for X, uber lyft, urban planning, Vannevar Bush, Vernor Vinge, wage slave

Figure 4.3: Uber’s homepage offers a message of simultaneous elitism and equality (image from July 2014). Source: Uber, http://mascola.com/insights/ubers-lost-positoning-luxury-car-service/. Figure 4.4: Lyft advertising takes a very different tack from Uber. Source: http://www.adweek.com/news/technology/lyft-hopes-accelerate-first-integrated-ad-campaign-159619. Figure 4.5: Amazon Mechanical Turk Interface for Managing Workers. © 2016, Amazon Web Services, Inc. or its affiliates. All rights reserved. http://docs.aws.amazon.com/AWSMechTurk/latest/RequesterUI/ViewingWorkerDetails.html. Figure 4.6: An engraving of the Turk from Karl Gottlieb von Windisch’s 1784 book Inanimate Reason. Figure 5.1: The Blockchain, a system for transparent, public accounting of Bitcoin transactions. Creative Commons: Matthaus Wander; https://commons.wikimedia.org/wiki/File:Bitcoin_Block_Data.png.


Industry 4.0: The Industrial Internet of Things by Alasdair Gilchrist

3D printing, additive manufacturing, Amazon Web Services, augmented reality, autonomous vehicles, barriers to entry, business intelligence, business process, chief data officer, cloud computing, connected car, cyber-physical system, deindustrialization, DevOps, digital twin, fault tolerance, global value chain, Google Glasses, hiring and firing, industrial robot, inflight wifi, Infrastructure as a Service, Internet of things, inventory management, job automation, low cost airline, low skilled workers, microservices, millennium bug, pattern recognition, peer-to-peer, platform as a service, pre–internet, race to the bottom, RFID, Skype, smart cities, smart grid, smart meter, smart transportation, software as a service, stealth mode startup, supply-chain management, trade route, undersea cable, web application, WebRTC, Y2K

For example, the type of product, the price, discount, and any calorific or nutrient data normally declared on the label. Knowing a user’s location, activity, and interests will enable location based services (LBS), such as instantaneously providing a coupon for a discount. The Cloud and Fog Cloud computing is similar to many technologies that have been around for decades. It really came to the fore, in the format that we now recognize, in the mid 2000s with the launch of Amazon Web Services (AWS). AWS was followed by RackSpace, Google’s CE, and Microsoft Azure, among several others. Amazon’s vision of the cloud was on hyper-provisioning; in so much as they built massive data centers with hyper-capacity in order to meet their web-scale requirements. Amazon then took the business initiative to rent spare capacity to other businesses, in the form of leasing compute, and storage resources on an as-used basis.


pages: 290 words: 87,549