load shedding

16 results back to index


pages: 719 words: 181,090

Site Reliability Engineering: How Google Runs Production Systems by Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy

"Margaret Hamilton" Apollo, Abraham Maslow, Air France Flight 447, anti-pattern, barriers to entry, business intelligence, business logic, business process, Checklist Manifesto, cloud computing, cognitive load, combinatorial explosion, continuous integration, correlation does not imply causation, crowdsourcing, database schema, defense in depth, DevOps, en.wikipedia.org, exponential backoff, fail fast, fault tolerance, Flash crash, George Santayana, Google Chrome, Google Earth, if you see hoof prints, think horses—not zebras, information asymmetry, job automation, job satisfaction, Kubernetes, linear programming, load shedding, loose coupling, machine readable, meta-analysis, microservices, minimum viable product, MVC pattern, no silver bullet, OSI model, performance metric, platform as a service, proprietary trading, reproducible builds, revision control, risk tolerance, side project, six sigma, the long tail, the scientific method, Toyota Production System, trickle-down economics, warehouse automation, web application, zero day

time requirements, Balance in Quality training for, Learning Paths That Are Cumulative and Orderly-A Hunger for Failure: Reading and Sharing Postmortems training materials, Creating Stellar Reverse Engineers and Improvisational Thinkers typical activities, Life of an On-Call Engineer one-phase pipelines, Initial Effect of Big Data on the Simple Pipeline Pattern open commenting/annotation system, Collaborate and Share Knowledge operational loadcross-industry lessons, Automating Away Repetitive Work and Operational Overhead managing, Managing Operational Load ongoing responsibilities, Managing Operational Load types of, Dealing with Interrupts operational overload, Operational Overload operational underload, A Treacherous Enemy: Operational Underload operational work (see toil) out-of-band checks and balances, Choosing a Strategy for Superior Data Integrity, Out-of-band data validation out-of-band communications systems, What went well outage trackingbaselines and progress tracking, Tracking Outages benefits of, Unexpected Benefits Escalator, Escalator Outalator, Outalator-Reporting and communication Outalatoraggregation in, Aggregation benefits of, Outalator building your own, Outalator incident analysis, Analysis notification process, Outalator reporting and communication, Reporting and communication tagging in, Tagging overhead, Toil Defined overload handlingapproaches to, Handling Overload best practices for, Overloads and Failure client-side throttling, Client-Side Throttling load from connections, Load from Connections overload errors, Handling Overload Errors overview of, Conclusions per-client retry budget, Deciding to Retry per-customer limits, Per-Customer Limits per-request retry budget, Deciding to Retry product launches and, Overload Behavior and Load Tests request criticality, Criticality retrying requests, Deciding to Retry(see also retries, RPC) utilization signals, Utilization Signals(see also cascading failures) P package managers, Packaging packet encapsulation, Load Balancing at the Virtual IP Address Paxos consensus algorithmClassic Paxos algorithm, Reasoning About Performance: Fast Paxos disk access and, Disk Access Egalitarian Paxos consensus algorithm, Stable Leaders Fast Paxos consensus algorithm, Reasoning About Performance: Fast Paxos, The Use of Paxos Lamport’s Paxos protocol, How Distributed Consensus Works(see also consensus algorithms) performanceefficiency and, Efficiency and Performance monitoring, Worrying About Your Tail (or, Instrumentation and Performance) performance tests, System tests periodic pipelines, Challenges with the Periodic Pipeline Pattern periodic scheduling (see cron) persistent storage, Disk Access Photon, Number of Replicas pipelining, Batching planned changes, Planned Changes, Drains, or Turndowns policies and procedures, enforcing, Enforcement of Policies and Procedures post hoc analysis, Setting Reasonable Expectations for Monitoring postmortemsbenefits of, Postmortem Culture: Learning from Failure best practices for, Google’s Postmortem Philosophy-Introducing a Postmortem Culture, Postmortems collaboration and sharing in, Collaborate and Share Knowledge concept of, Postmortem Culture: Learning from Failure cross-industry lessons, Postmortem Culture-Postmortem Culture example postmortem, Example Postmortem-Timeline formal review and publication of, Collaborate and Share Knowledge Google's philosophy for, Google’s Postmortem Philosophy guidelines for, Ensuring a Durable Focus on Engineering introducing postmortem cultures, Introducing a Postmortem Culture on-call engineering and, A Hunger for Failure: Reading and Sharing Postmortems ongoing improvements to, Conclusion and Ongoing Improvements rewarding participation in, Introducing a Postmortem Culture triggers for, Google’s Postmortem Philosophy privacy, Choosing a Strategy for Superior Data Integrity proactive testing, Encourage Proactive Testing problem reports, Problem Report process death, Process Death process health checks, Stop Health Check Failures/Deaths process updates, Process Updates process-induced emergencies, Process-Induced Emergency Prodtest (Production Test), Detecting Inconsistencies with Prodtest product launchesbest practices for, Progressive Rollouts defined, Reliable Product Launches at Scale development of Launch Coordination Engineering (LCE), Development of LCE-Infrastructure churn driving convergence and simplification, Driving Convergence and Simplification launch coordination checklists, The Launch Checklist-Example action items, Launch Coordination Checklist launch coordination engineering, Launch Coordination Engineering NORAD Tracks Santa example, Reliable Product Launches at Scale overview of, Conclusion processes for, Setting Up a Launch Process rate of, Reliable Product Launches at Scale techniques for reliable, Selected Techniques for Reliable Launches-Overload Behavior and Load Tests production environment (see Google production environment) production inconsistenciesdetecting with Prodtest, Detecting Inconsistencies with Prodtest resolving idempotently, Resolving Inconsistencies Idempotently production meetings, Communications: Production Meetings-Attendanceagenda example, Example Production Meeting Minutes production probes, Production Probes Production Readiness Review process (see SRE engagement model) production tests, Production Tests protocol buffers (protobufs), Our Software Infrastructure, Integration Protocol Data Units, Load Balancing at the Virtual IP Address provisioning, guidelines for, Provisioning PRR (Production Readiness Review) model, The PRR Model, Production Readiness Reviews: Simple PRR Model-Continuous Improvement push frequency, Motivation for Error Budgets push managers, Ongoing responsibilities Python’s safe_load, Integration Q “queries per second” model, The Pitfalls of “Queries per Second” Query of Death, Process Death queuingcontrolled delay, Load Shedding and Graceful Degradation first-in, first-out, Load Shedding and Graceful Degradation last-in, first-out, Load Shedding and Graceful Degradation management of, Queue Management, Reliable Distributed Queuing and Messaging queuing-as-work-distribution pattern, Reliable Distributed Queuing and Messaging quorum (see distributed consensus systems) R Raft consensus protocol, Multi-Paxos: Detailed Message Flow, Stable Leaders(see also consensus algorithms) RAID, Overarching Layer: Replication Rapid automated release system, Continuous Build and Deployment, Rapid read workload, scaling, Scaling Read-Heavy Workloads real backups, Backups Versus Archives real-time collaboration, Collaborate and Share Knowledge recoverability, Challenges of Maintaining Data Integrity Deep and Wide recovery, Knowing That Data Recovery Will Work recovery systems, Delivering a Recovery System, Rather Than a Backup System recursion (see recursion) recursive DNS servers, Load Balancing Using DNS recursive separation of responsibilities, Recursive Separation of Responsibilities redundancy, Challenges of Maintaining Data Integrity Deep and Wide, Overarching Layer: Replication Reed-Solomon erasure codes, Overarching Layer: Replication regression tests, System tests release engineeringchallenges of, Release Engineering continuous build and deployment, Continuous Build and Deployment-Configuration Management defined, Release Engineering instituting, Start Release Engineering at the Beginning philosophy of, Philosophy-Enforcement of Policies and Procedures the role of release engineers, The Role of a Release Engineer wider application of, Conclusions reliability testingamount required, Testing for Reliability benefits of, Conclusion break-glass mechanisms, Expect Testing Fail canary tests, Canary test configuration tests, Configuration test coordination of, The Need for Speed creating test and build environments, Creating a Test and Build Environment error budgets, Pursuing Maximum Change Velocity Without Violating a Service’s SLO, Motivation for Error Budgets-Forming Your Error Budget, Error Budgets expecting test failure, Expect Testing Fail-Expect Testing Fail fake backend versions, Production Probes goals of, Testing for Reliability importance of, Preface integration tests, Integration tests, Integration MTTR and, Testing for Reliability performance tests, System tests proactive, Encourage Proactive Testing production probes, Production Probes production tests, Production Tests regression tests, System tests reliability goals, Embracing Risk sanity testing, System tests segregated environments and, Pushing to Production smoke tests, System tests speed of, The Need for Speed statistical tests, Testing Disaster stress tests, Stress test system tests, System tests testing at scale, Testing at Scale-Production Probes timing of, Production Tests unit tests, Unit tests reliable replicated datastores, Reliable Replicated Datastores and Configuration Stores Remote Procedure Call (RPC), Our Software Infrastructure, Examine, Criticalitybimodal, Bimodal latency deadlinesmissing, Missing deadlines propagating, Load Shedding and Graceful Degradation, Deadline propagation queue management, Queue Management, Reliable Distributed Queuing and Messaging selecting, Latency and Deadlines retries, Retries-Retries RPC criticality, Criticality(see also overload handling) replicasadding, Capacity and Load Balancing drawbacks of leader replicas, Capacity and Load Balancing location of, Location of Replicas, Quorum composition number deployed, Number of Replicas replicated logs, Number of Replicas replicated state machine (RSM), Reliable Replicated State Machines replication, Challenges of Maintaining Data Integrity Deep and Wide, Overarching Layer: Replication request latency, Indicators, The Four Golden Signals request profile changes, Request profile changes request success rate, Measuring Service Risk resilience testing, Practices resourcesallocation of, Hardware, Managing Machines exhaustion, Resource Exhaustion limits, Resource limits(see also capacity planning) restores, 1T Versus 1E: Not “Just” a Bigger Backup retention, Retention retries, RPCavoiding, Deciding to Retry cascading failures due to, Retries considerations for automatic, Retries diagnosing outages due to, Retries handling overload errors and, Handling Overload Errors per-client retry budgets, Deciding to Retry per-request retry budgets, Deciding to Retry reverse engineering, Reverse Engineers: Figuring Out How Things Work reverse proxies, What went well revision history, First Layer: Soft Deletion risk managementbalancing risk and innovation, Embracing Risk costs of, Managing Risk error budgets, Motivation for Error Budgets-Benefits, Error Budgets key insights, Benefits measuring service risk, Measuring Service Risk risk tolerance of services, Risk Tolerance of Services-Example: Frontend infrastructure rollback procedures, What we learned rollouts, New Rollouts, Rollout Planning, Progressive Rollouts root causeanalysis of, Practices, Google’s Postmortem Philosophy(see also postmortems) defined, Definitions Round Robin policy, Simple Round Robin round-trip-time (RTT), Distributed Consensus Performance and Network Latency rows, Hardware rule evaluation, in monitoring systems, Rule Evaluation-Rule Evaluation S Safari® Books Online, Safari® Books Online sanity testing, System tests saturation, The Four Golden Signals scaledefined, Choosing a Strategy for Superior Data Integrity issues in, Scaling issues: Fulls, incrementals, and the competing forces of backups and restores securityin release engineering, Enforcement of Policies and Procedures new approach to, Practices self-service model, Self-Service Model separation of responsibilities, Recursive Separation of Responsibilities serversvs. clients, Our Software Infrastructure defined, Hardware overload scenario, Server Overload preventing overload, Preventing Server Overload-Always Go Downward in the Stack service availabilityavailability table, Availability Table cost factors, Cost, Cost defined, Indicators target for consumer services, Target level of availability target for infrastructure service, Target level of availability time-based equation, Measuring Service Risk types of consumer service failures, Types of failures types of infrastructure services failures, Types of failures service health checks, Stop Health Check Failures/Deaths service latencylooser approach to, Other service metrics monitoring for, The Four Golden Signals service level agreements (SLAs), Agreements service level indicators (SLIs)aggregating raw measurements, Aggregation collecting indicators, Collecting Indicators defined, Indicators standardizing indicators, Standardize Indicators service level objectives (SLOs)agreements in practice, Agreements in Practice best practices for, Define SLOs Like a User choosing, Service Level Objectives-Objectives control measures, Control Measures defined, Objectives defining objectives, Objectives in Practice selecting relevant indicators, What Do You and Your Users Care About?

For details, see “Testing for Cascading Failures”. Serve degraded results Serve lower-quality, cheaper-to-compute results to the user. Your strategy here will be service-specific. See “Load Shedding and Graceful Degradation”. Instrument the server to reject requests when overloaded Servers should protect themselves from becoming overloaded and crashing. When overloaded at either the frontend or backend layers, fail early and cheaply. For details, see “Load Shedding and Graceful Degradation”. Instrument higher-level systems to reject requests, rather than overloading servers Note that because rate limiting often doesn’t take overall service health into account, it may not be able to stop a failure that has already begun.

For example, Gmail often uses queueless servers, relying instead on failover to other server tasks when the threads are full. On the other end of the spectrum, systems with “bursty” load for which traffic patterns fluctuate drastically may do better with a queue size based on the current number of threads in use, processing time for each request, and the size and frequency of bursts. Load Shedding and Graceful Degradation Load shedding drops some proportion of load by dropping traffic as the server approaches overload conditions. The goal is to keep the server from running out of RAM, failing health checks, serving with extremely high latency, or any of the other symptoms associated with overload, while still doing as much useful work as it can.


pages: 1,380 words: 190,710

Building Secure and Reliable Systems: Best Practices for Designing, Implementing, and Maintaining Systems by Heather Adkins, Betsy Beyer, Paul Blankinship, Ana Oprea, Piotr Lewandowski, Adam Stubblefield

air gap, anti-pattern, barriers to entry, bash_history, behavioural economics, business continuity plan, business logic, business process, Cass Sunstein, cloud computing, cognitive load, continuous integration, correlation does not imply causation, create, read, update, delete, cryptocurrency, cyber-physical system, database schema, Debian, defense in depth, DevOps, Edward Snowden, end-to-end encryption, exponential backoff, fault tolerance, fear of failure, general-purpose programming language, Google Chrome, if you see hoof prints, think horses—not zebras, information security, Internet of things, Kubernetes, load shedding, margin call, microservices, MITM: man-in-the-middle, NSO Group, nudge theory, operational security, performance metric, pull request, ransomware, reproducible builds, revision control, Richard Thaler, risk tolerance, self-driving car, single source of truth, Skype, slashdot, software as a service, source of truth, SQL injection, Stuxnet, the long tail, Turing test, undersea cable, uranium enrichment, Valgrind, web application, Y2K, zero day

Figure 8-3 also distinguishes the uncontrolled nature of degraded traffic (the backward-slashed area) prior to system crash. Figure 8-4 shows that the system with load shedding rejects significantly less traffic than in Figure 8-3 (the crosshatched area), with the rest of the traffic either processed without failure (whitespace area) or rejected if lower priority (forward-slashed area). Figure 8-3. Complete outage and a possible cascading failure from a load spike Figure 8-4. Using load shedding and throttling to manage a load spike Load shedding The primary resilience objective of load shedding (described in Chapter 22 of the SRE book) is to stabilize components at maximum load, which can be especially beneficial for preserving security-critical functions.

Creating a safeguard by permanently overprovisioning servers wastes money and doesn’t guarantee a safe response. Instead, servers should adjust how they respond to load based upon current conditions. You can use two specific automation strategies here: Load shedding is done by returning errors rather than serving requests. Throttling of clients is done by delaying responses until closer to the request deadline. Figure 8-3 illustrates a traffic spike that exceeds the capacity. Figure 8-4 illustrates the effects of load shedding and throttling to manage the load spike. Note the following: The curve represents requests per second, and the area under it represents total requests. Whitespace represents traffic processed without failure.

This approach reduces the rate of requests the server receives from clients (if clients send requests sequentially), which means that you can redirect the resources saved during wait times. Similar to load shedding, you could define policies to apply throttling to specific offending clients, or more generally to all clients. Request priority and cost play a role in selecting which requests to throttle. Automated response Server utilization statistics can help determine when to consider applying controls like load shedding and throttling. The more heavily a server is loaded, the less traffic or load it can handle. If controls take too long to activate, higher-priority requests may end up being dropped or throttled.


pages: 376 words: 101,759

Shorting the Grid: The Hidden Fragility of Our Electric Grid by Meredith. Angwin

airline deregulation, California energy crisis, carbon credits, carbon footprint, congestion pricing, corporate governance, Credit Default Swap, crony capitalism, David Brooks, decarbonisation, demand response, distributed generation, electricity market, en.wikipedia.org, energy security, green new deal, Hans Rosling, Intergovernmental Panel on Climate Change (IPCC), invisible hand, Jones Act, Just-in-time delivery, load shedding, market clearing, Michael Shellenberger, Negawatt, off-the-grid, performance metric, plutocrats, renewable energy credits, rolling blackouts, Silicon Valley, smart grid, smart meter, the map is not the territory, Tragedy of the Commons, uranium enrichment, vertical integration, washing machines reduced drudgery, zero-sum game

In some states, RECs can be purchased from distant states. Restructured states or areas: States (or areas within states) that have joined an RTO market. RMR: Reliability Must Run. Rolling blackouts: A form of load shedding, in which a utility schedules power outages for one area and then returns the power to that area and schedules a power outage for another area. The blackout (load shedding) “rolls” from place to place. See Load shedding. RTO: Regional Transmission Organization. Southwest Power Pool (SPP): The Regional Transmission Organization whose territory covers much of the Midwest and some areas of the South. SPP territories lie mostly to the west of MISO territories.

Nobody is responsible IN CALIFORNIA, AS ELSEWHERE, the distribution utilities were responsible for keeping their lines and substations running. The lines and substations ran fine. The RTO (CAISO) was responsible for dispatching the plants that were available for dispatch. If too many of them were offline for “maintenance” and there had to be “load shedding” (rolling blackouts), well, CAISO is not responsible for plant maintenance, just for dispatch. Every individual plant makes money when it runs. If a plant needs maintenance or whatever at an awkward time, it’s not making money at that time. That individual plant is not responsible for keeping the grid going.

In early April, Chicago-based energy provider Exelon Corp. said it would close two large natural-gas units at Mystic Station, Massachusetts. In its report about possibilities for the winter of 2024–25, ISO-NE had included the loss of these two plants in one of its scenarios. The ISO-NE report concluded that Mystic’s possible closure would cause 20 to 50 hours of “load shedding” (meaning rolling blackouts) and hundreds of hours of grid operation under emergency protocols. When Exelon made its closure announcement, ISO-NE realized that the danger of rolling blackouts was suddenly more immediate than 2024. It now hopes to provide “out-of-market-cost recovery” — subsidies — to persuade Exelon to keep the Mystic plants operating.


pages: 757 words: 193,541

The Practice of Cloud System Administration: DevOps and SRE Practices for Web Services, Volume 2 by Thomas A. Limoncelli, Strata R. Chalup, Christina J. Hogan

active measures, Amazon Web Services, anti-pattern, barriers to entry, business process, cloud computing, commoditize, continuous integration, correlation coefficient, database schema, Debian, defense in depth, delayed gratification, DevOps, domain-specific language, en.wikipedia.org, fault tolerance, finite state, Firefox, functional programming, Google Glasses, information asymmetry, Infrastructure as a Service, intermodal, Internet of things, job automation, job satisfaction, Ken Thompson, Kickstarter, level 1 cache, load shedding, longitudinal study, loose coupling, machine readable, Malcom McLean invented shipping containers, Marc Andreessen, place-making, platform as a service, premature optimization, recommendation engine, revision control, risk tolerance, Salesforce, scientific management, seminal paper, side project, Silicon Valley, software as a service, sorting algorithm, standardized shipping container, statistical model, Steven Levy, supply-chain management, systems thinking, The future is already here, Toyota Production System, vertical integration, web application, Yogi Berra

Additional capacity can be found at other service providers as well. A public cloud computing provider can be used as the shared pool. Usually you will not have to pay for unused capacity. Shared resource pools are not just appropriate for machines, but may also be used for storage and other resources. Load Shedding Another strategy is load shedding. With this strategy the service turns away some users so that other users can have a good experience. To make an analogy, an overloaded phone system doesn’t suddenly disconnect all existing calls. Instead, it responds to any new attempts to make a call with a “fast busy” tone so that the person will try to make the call later.

A variation of load shedding is stopping certain tasks that can be put off until later. For example, low-priority database updates could be queued up for processing later; a social network that stores reputation points for users might store the fact that points have been awarded rather than processing them; nightly bulk file transfers might be delayed if the network is overloaded. That said, tasks that can be put off for a couple of hours might cause problems if they are put off forever. There is, after all, a reason they exist. For any activity that is delayed due to load shedding, there must be a plan on how such a delay is handled.

Establish a service level agreement (SLA) to determine how long something can be delayed and to identify a timeline of actions that should be undertaken to mitigate problems or extend the deadlines. Low-priority updates might become a high priority after a certain amount of time. If many systems are turned off due to load shedding, it might be possible to enable them, one at a time, to let each catch up. To be able to manage such situations one must have visibility into the system so that prioritization decisions can be made. For example, knowing the age of a task (how long it has been delayed), predicting how long it will take to process, and indicating how close it is to a deadline will permit operations personnel to gauge when delayed items should be continued


pages: 297 words: 95,518

Ten Technologies to Save the Planet: Energy Options for a Low-Carbon Future by Chris Goodall

barriers to entry, carbon footprint, carbon tax, congestion charging, decarbonisation, electricity market, energy security, Indoor air pollution, Intergovernmental Panel on Climate Change (IPCC), Kickstarter, land tenure, load shedding, New Urbanism, oil shock, profit maximization, Silicon Valley, smart grid, smart meter, statistical model, undersea cable

To the electricity provider, these reductions may be equivalent to having an extra power station available; however, a spare power station that sits idle 90 percent of the time would be a much more expensive and polluting solution. The major load-shedding programs around the world typically cover about 2 percent of peak power use, enough to cope with temporary energy deficiencies as long as wind is not too great a percentage of total electricity supply. For Spain and Denmark, countries where wind occasionally provides a very large fraction of total electricity demand for several hours at a time, this scheme would not be enough on its own. However, many people in the electricity industry think that load-shedding programs could cover 10 percent of peak power use or even more.

Otherwise, the voltage or frequency of the alternating current would move outside the tolerances of home appliances and business equipment, possibly causing damage. Pumped storage is a way of quickly adjusting supply, but there are also methods to almost instantaneously reduce demand. In the jargon of the electricity industry, this activity is known as “load shedding.” Some manufacturing companies, for example, have agreements that allow their electricity supplier to disconnect them at a few moments’ notice. In return, they pay lower prices for their electricity. This system works well. Although it is designed primarily to shave a little off the sharp daily peaks of electricity demand, there is no reason why the same approach couldn’t be adapted to deal with temporary shortfalls in wind power at all times of day.

A signal carried over the mobile phone network might trigger an electronic on/off switch at the wall socket of those electric appliances that use large amounts of electricity. A program involving all three of these techniques for balancing supply and demand—better electricity grids, greater energy storage, and load shedding—allows electricity operators to deal with the occasional unexpected drop in wind power. There will be some costs, and some inconvenience, such as washing machines unexpectedly switching off for an hour or so, but it can be done. In many parts of the world, regional electricity grids are already finding ways to reduce peak demand.


pages: 444 words: 118,393

The Nature of Software Development: Keep It Simple, Make It Valuable, Build It Piece by Piece by Ron Jeffries

Amazon Web Services, anti-pattern, bitcoin, business cycle, business intelligence, business logic, business process, c2.com, call centre, cloud computing, continuous integration, Conway's law, creative destruction, dark matter, data science, database schema, deep learning, DevOps, disinformation, duck typing, en.wikipedia.org, fail fast, fault tolerance, Firefox, Hacker News, industrial robot, information security, Infrastructure as a Service, Internet of things, Jeff Bezos, Kanban, Kubernetes, load shedding, loose coupling, machine readable, Mars Rover, microservices, Minecraft, minimum viable product, MITM: man-in-the-middle, Morris worm, move fast and break things, OSI model, peer-to-peer lending, platform as a service, power law, ransomware, revision control, Ruby on Rails, Schrödinger's Cat, Silicon Valley, six sigma, software is eating the world, source of truth, SQL injection, systems thinking, text mining, time value of money, transaction costs, Turing machine, two-pizza team, web application, zero day

The caller will stop waiting for a response on the original request and probably fire a retry at us (exactly when it hurts the worst!). Preventing Disaster With that perspective, we can see that the best thing to do under high load is turn away work we can’t complete in time. This is called “load shedding,” and it’s the most important way to control incoming demand. Load shedding happens very quickly when a socket’s listen queue is full, and a quick rejection is better than a slow timeout. More generally, we want to shed load as early as possible so we can avoid tying up resources at several tiers before rejecting the request. Load balancers near the network edge are the ideal place.

At some point, though, the API server still has a thread waiting on a call. As we saw in ​Blocked Threads​, blocked threads are a quick path to downtime. At the edge of your system boundary, blocked threads will frustrate a user or provoke a retry loop. As such, back pressure works best within a system boundary. At the edges, you also need load shedding and asynchronous calls. In our example, the API server should accept calls on one thread pool and then issue the outbound call to storage on another set of threads. That way, when the outbound call blocks, the request-handling thread can time out, unblock, and respond with an HTTP 503. Alternatively, it could drop a “create tag” command in a queue for later indexing.

That way you can tell whether it’s a random fluctuation or a trend. Remember This Back Pressure creates safety by slowing down consumers. Consumers will experience slowdowns. The only alternative is to let them crash the provider. Apply Back Pressure within a system boundary Across boundaries, look at load shedding instead. This is especially true when the Internet at large is your user base. Queues must be finite for response times to be finite. You only have a few options when a queue is full. All of them are unpleasant: drop data, refuse work, or block. Consumers must be careful not to block forever.


pages: 464 words: 127,283

Smart Cities: Big Data, Civic Hackers, and the Quest for a New Utopia by Anthony M. Townsend

1960s counterculture, 4chan, A Pattern Language, Adam Curtis, air gap, Airbnb, Amazon Web Services, anti-communist, Apple II, Bay Area Rapid Transit, Big Tech, bike sharing, Boeing 747, Burning Man, business process, call centre, carbon footprint, charter city, chief data officer, clean tech, clean water, cloud computing, company town, computer age, congestion charging, congestion pricing, connected car, crack epidemic, crowdsourcing, DARPA: Urban Challenge, data acquisition, Deng Xiaoping, digital divide, digital map, Donald Davies, East Village, Edward Glaeser, Evgeny Morozov, food desert, game design, garden city movement, General Motors Futurama, gentrification, Geoffrey West, Santa Fe Institute, George Gilder, ghettoisation, global supply chain, Grace Hopper, Haight Ashbury, Hedy Lamarr / George Antheil, Herman Kahn, hive mind, Howard Rheingold, interchangeable parts, Internet Archive, Internet of things, Jacquard loom, Jane Jacobs, Jevons paradox, jitney, John Snow's cholera map, Joi Ito, Khan Academy, Kibera, Kickstarter, knowledge worker, Lewis Mumford, load shedding, lolcat, M-Pesa, machine readable, Mark Zuckerberg, megacity, megaproject, messenger bag, mobile money, mutually assured destruction, new economy, New Urbanism, Norbert Wiener, Occupy movement, off grid, One Laptop per Child (OLPC), openstreetmap, packet switching, PalmPilot, Panopticon Jeremy Bentham, Parag Khanna, patent troll, Pearl River Delta, place-making, planetary scale, popular electronics, power law, RFC: Request For Comment, RFID, ride hailing / ride sharing, Robert Gordon, scientific management, self-driving car, sharing economy, Shenzhen special economic zone , Silicon Valley, SimCity, Skype, smart cities, smart grid, smart meter, social graph, social software, social web, SpaceShipOne, special economic zone, Steve Jobs, Steve Wozniak, Stuxnet, supply-chain management, technoutopianism, Ted Kaczynski, telepresence, The Death and Life of Great American Cities, too big to fail, trade route, Twitter Arab Spring, Tyler Cowen, Tyler Cowen: Great Stagnation, undersea cable, Upton Sinclair, uranium enrichment, urban decay, urban planning, urban renewal, Vannevar Bush, working poor, working-age population, X Prize, Y2K, zero day, Zipcar

Smart meters that communicate directly with smart appliances might automatically reschedule a load of wash for later in the day when demand and prices are likely to fall. Even the most sophisticated load-shifting scheme will one day meet its limit. That’s when utilities wield their trump card—load shedding—a kind of targeted blackout. Traditionally, load shedding was a manual process. Utilities would cut deals with large users of electricity like factories and universities to shut down power during peaking crises in return for a discount on their regular rates. Smart meters will allow these miniblackouts to be replaced by sophisticated surgical drawdowns on sacrificial facilities and equipment.

While peaking plants can also be highly efficient—most are natural-gas–powered turbines—they are far more costly per unit of power to build and run. If only the peaks could be evened out, fewer peaking plants would be needed and utilities could focus more on ruthlessly fine-tuning base load plants to be as lean and clean as possible.48 Smart grids offer two tricks to even out the peaks: load shifting and load shedding. Load shifting, the gentler of the two, tries to spread demand for power away from peak periods of demand through price incentives. In their simplest form, smart meters allow businesses and consumers to see the true cost of generating electricity during periods of high demand. As they fire up those costly peaking plants, utilities simply pass the higher generating cost along to consumers.

The wiggle room that once existed in the form of reserve generating capacity is fast disappearing, raising the possibility of regular blackouts in the future. During the 1990s, demand for electricity grew by 35 percent in the United States, but generating capacity increased by only 18 percent.49 According to Siemens, smart grids will help utility engineers sleep at night, since load shedding and load shifting could reduce national electricity needs by up to 10 percent. 50 Environmentalists will cheer because improved demand management removes a key obstacle to greater reliance on renewable generating sources, which are notoriously unreliable base capacity—the sun doesn’t always shine and the wind doesn’t always blow.


pages: 540 words: 103,101

Building Microservices by Sam Newman

airport security, Amazon Web Services, anti-pattern, business logic, business process, call centre, continuous integration, Conway's law, create, read, update, delete, defense in depth, don't repeat yourself, Edward Snowden, fail fast, fallacies of distributed computing, fault tolerance, index card, information retrieval, Infrastructure as a Service, inventory management, job automation, Kubernetes, load shedding, loose coupling, microservices, MITM: man-in-the-middle, platform as a service, premature optimization, pull request, recommendation engine, Salesforce, SimCity, social graph, software as a service, source of truth, sunk-cost fallacy, systems thinking, the built environment, the long tail, two-pizza team, web application, WebSocket

Timeouts and circuit breakers help you free up resources when they are becoming constrained, but bulkheads can ensure they don’t become constrained in the first place. Hystrix allows you, for example, to implement bulkheads that actually reject requests in certain conditions to ensure that resources don’t become even more saturated; this is known as load shedding. Sometimes rejecting a request is the best way to stop an important system from becoming overwhelmed and being a bottleneck for multiple upstream services. Isolation The more one service depends on another being up, the more the health of one impacts the ability of the other to do its job.

Index A acceptance testing, Types of Tests access by reference, Access by Reference accountability, People adaptability, Summary Aegisthus project, Backup Data Pump aggregated logs, Logs, Logs, and Yet More Logs… antifragile systems, Microservices, The Antifragile Organization-Isolationbulkheads, Bulkheads circuit breakers, Circuit Breakers examples of, The Antifragile Organization increased use of, Microservices isolation, Isolation load shedding, Bulkheads timeouts, Timeouts AP systemdefinition of term, Sacrificing Consistency vs. CP system, AP or CP? API key-based authentication, API Keys, It’s All About the Keys application containers, Application Containers architects (see systems architects) architectural principlesdevelopment of, Principles Heroku's 12 factors, Principles key microservices principles, Bringing It All Together real-world example, A Real-World Example architectural safety, Architectural Safety, Architectural Safety Measures artifactsimages, Images as Artifacts operating system, Operating System Artifacts platform-specific, Platform-Specific Artifacts asynchronous collaborationcomplexities of, Complexities of Asynchronous Architectures implementing, Implementing Asynchronous Event-Based Collaboration vs. synchronous, Synchronous Versus Asynchronous ATOM specification, Technology Choices authentication/authorization, Authentication and Authorization-The Deputy Problemdefinition of terms, Authentication and Authorization fine-grained, Fine-Grained Authorization service-to-service, Service-to-Service Authentication and Authorization single sign-on (SSO), Common Single Sign-On Implementations single sign-on gateway, Single Sign-On Gateway terminology, Common Single Sign-On Implementations automationbenefits for deployment, Automation case studies on, Two Case Studies on the Power of Automation autonomymicroservices and, Autonomous role of systems architect in, Summary autoscaling, Autoscaling availabilityin CAP theorem, CAP Theorem key microservices principle of, How Much Is Too Much?

JSON web tokens (JWT), HMAC Over HTTP K Karyon, Tailored Service Template key-based authentication, API Keys Kibana, Logs, Logs, and Yet More Logs… L latency, How Much Is Too Much? Latency Monkey, The Antifragile Organization layered architectures, Microservices librariesclient, Client Libraries service metrics, Service Metrics shared, Shared Libraries Linux containers, Linux Containers load balancing, Load Balancing load shedding, Bulkheads local calls, Local Calls Are Not Like Remote Calls logsaggregated, Logs, Logs, and Yet More Logs…(see also monitoring) security issues, Logging standardization of, Standardization logstash, Logs, Logs, and Yet More Logs… loose coupling, Loose Coupling, Orchestration Versus Choreography, Loose and Tightly Coupled Organizations M man-in-the-middle attacks, Allow Everything Inside the Perimeter Marick's quadrant, Types of Tests maturity, Maturity mean time between failures (MTBF), Mean Time to Repair Over Mean Time Between Failures?


Nepal Travel Guide by Lonely Planet

call centre, carbon footprint, clean water, happiness index / gross national happiness, illegal immigration, Kickstarter, land reform, load shedding, Mahatma Gandhi, mass immigration, Skype, sustainable-tourism, trade route, traffic fines

Avoid marches, demonstrations and disturbances, as they can quickly turn violent. The website www.nepalbandh.com warns of any upcoming strikes. When roads are closed the government generally runs buses with armed police from the airport to major hotels, returning to the airport from Tridevi Marg at the east end of Thamel. Load Shedding Electricity cuts (‘load shedding’) are a fact of life in Kathmandu and Pokhara, especially in winter, when water and thus hydro power levels are at their lowest. Electricity is currently rationed across Kathmandu, shifting from district to district every eight hours or so. Most hotels post a schedule of planned electricity cuts, which can last up to 16 hours a day in both Kathmandu and Pokhara.

Information Emergency Ambulance service ( 4521048) Provided by Patan Hospital. Fire Brigade ( 101, 4221177) Police ( 100, 4223011; www.nepalpolice.gov.np; Durbar Sq) Red Cross Ambulance ( 4228094) Tourist Police Bhrikuti Mandap ( 4247041); Thamel ( 4700750) Dangers & Annoyances Electricity cuts (‘load shedding’) are a fact of life in Kathmandu; they last for up to 16 hours a day in winter when hydro power levels are at their lowest. Electricity is currently rationed across the city, shifting from district to district every eight hours or so. Most hotels post a schedule of planned electricity cuts. Try to choose a hotel with a generator and make sure your room is far away from it.

Electricity Electricity is 220V/50 cycles; 120V appliances from the USA will need a transformer. Sockets usually take plugs with three round pins, sometimes the small variety, sometimes the large. Some sockets take plugs with two round pins. Local electrical shops sell cheap adapters. Blackouts (‘load shedding’) are a fact of life across Nepal, especially in Kathmandu; these peak in February with up to 16 hours a day of cuts. Power surges are also likely, so bring a voltage guard with spike suppressor (automatic cut-off switch) for your laptop. Embassies & Consulates Travellers continuing beyond Nepal may need visas for Bangladesh, China, India, Myanmar (Burma) and Thailand.


pages: 403 words: 105,550

The Key Man: The True Story of How the Global Elite Was Duped by a Capitalist Fairy Tale by Simon Clark, Will Louch

"Friedman doctrine" OR "shareholder theory", "World Economic Forum" Davos, An Inconvenient Truth, anti-communist, Berlin Wall, Bernie Madoff, British Empire, clean water, collapse of Lehman Brothers, colonial rule, coronavirus, corporate governance, COVID-19, dark triade / dark tetrad, do well by doing good, Donald Trump, fake news, forensic accounting, high net worth, impact investing, income inequality, Jeffrey Epstein, Kickstarter, load shedding, low cost airline, Mahatma Gandhi, megacity, Menlo Park, Michael Milken, Mohammed Bouazizi, Nelson Mandela, offshore financial centre, planetary scale, plutocrats, Ponzi scheme, profit maximization, rolling blackouts, Ronald Reagan, shareholder value, Silicon Valley, Social Responsibility of Business Is to Increase Its Profits, SoftBank, sovereign wealth fund, Suez crisis 1956, TED Talk, The Fortune at the Bottom of the Pyramid, trade route, Virgin Galactic, WikiLeaks, young professional

Her officials called Karachi Electric’s chief executive, Naveed Ismail, and listened carefully to what he had to say. Naveed said that Karachi Electric was controlled by Abraaj and required billions of dollars of loans and investment to improve its operations and reduce power cuts, which he called “load shedding.” After the meeting, Ambassador Patterson sent a cable to State Department colleagues asking for help to encourage Abraaj to raise the required funds for Karachi Electric. “Black outs and load shedding are a serious impediment to economic productivity. Poor power delivery has also led to large public demonstrations,” the American diplomatic cable said. “Embassy Islamabad requests the Department seek Consulate Dubai’s assistance.”


pages: 777 words: 186,993

Imagining India by Nandan Nilekani

"World Economic Forum" Davos, addicted to oil, affirmative action, Airbus A320, BRICs, British Empire, business process, business process outsourcing, call centre, carbon credits, carbon tax, clean water, colonial rule, corporate governance, cuban missile crisis, deindustrialization, demographic dividend, demographic transition, Deng Xiaoping, digital map, distributed generation, electricity market, farmers can use mobile phones to check market prices, flag carrier, full employment, ghettoisation, glass ceiling, global supply chain, Hernando de Soto, income inequality, informal economy, information asymmetry, Intergovernmental Panel on Climate Change (IPCC), joint-stock company, knowledge economy, land reform, light touch regulation, LNG terminal, load shedding, low cost airline, Mahatma Gandhi, market fragmentation, mass immigration, Mikhail Gorbachev, Network effects, new economy, New Urbanism, open economy, Parag Khanna, pension reform, Potemkin village, price mechanism, public intellectual, race to the bottom, rent control, rolodex, Ronald Reagan, school vouchers, Silicon Valley, smart grid, special economic zone, The Wealth of Nations by Adam Smith, Thomas L Friedman, Thomas Malthus, Tragedy of the Commons, transaction costs, trickle-down economics, unemployed young men, upwardly mobile, urban planning, urban renewal, women in the workforce, working poor, working-age population

There is no escaping our infrastructure problems, even here in India’s capital city—the Delhi newspapers during my visit have been full of headlines on the city’s power outages. A single day that week, as one outraged journalist wrote, had seen “10 periods of ‘load-shedding.’” In fact this particular bit of bureaucratspeak we use, “load-shedding,” reveals how a growing economy has found it difficult to look its crisis in the face.bt Our bad roads and power cuts are a reminder of our prereform years—it is here that we can most clearly see the evidence of India’s old structures, the tattered vestiges of socialism in an emerging free-market economy.

br It is a different story that this rule has often been violated by state governments. bs Bangalore can lay claim to pioneering bond issues by cities in India—India’s first such bond issue was by the Bangalore Mahanagarapalika in 1997 of Rs 1 billion, with a coupon rate of 13 percent. bt As a technical term, “load-shedding” means power cuts to tackle spikes in excess demand. But what India faces is consistent and severe power shortage. bu This view of roads and railways as an investment toward safety—to move people and goods in and out quickly, and avoid being cornered by enemies—has plenty of precedent. The Romans built Britain’s major road systems when they had occupied the restive island, and many of these still exist.


pages: 402 words: 110,972

Nerds on Wall Street: Math, Machines and Wired Markets by David J. Leinweber

"World Economic Forum" Davos, AI winter, Alan Greenspan, algorithmic trading, AOL-Time Warner, Apollo 11, asset allocation, banking crisis, barriers to entry, Bear Stearns, Big bang: deregulation of the City of London, Bob Litterman, book value, business cycle, butter production in bangladesh, butterfly effect, buttonwood tree, buy and hold, buy low sell high, capital asset pricing model, Charles Babbage, citizen journalism, collateralized debt obligation, Cornelius Vanderbilt, corporate governance, Craig Reynolds: boids flock, creative destruction, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, Danny Hillis, demand response, disintermediation, distributed generation, diversification, diversified portfolio, electricity market, Emanuel Derman, en.wikipedia.org, experimental economics, fake news, financial engineering, financial innovation, fixed income, Ford Model T, Gordon Gekko, Hans Moravec, Herman Kahn, implied volatility, index arbitrage, index fund, information retrieval, intangible asset, Internet Archive, Ivan Sutherland, Jim Simons, John Bogle, John Nash: game theory, Kenneth Arrow, load shedding, Long Term Capital Management, machine readable, machine translation, Machine translation of "The spirit is willing, but the flesh is weak." to Russian and back, market fragmentation, market microstructure, Mars Rover, Metcalfe’s law, military-industrial complex, moral hazard, mutually assured destruction, Myron Scholes, natural language processing, negative equity, Network effects, optical character recognition, paper trading, passive investing, pez dispenser, phenotype, prediction markets, proprietary trading, quantitative hedge fund, quantitative trading / quantitative finance, QWERTY keyboard, RAND corporation, random walk, Ray Kurzweil, Reminiscences of a Stock Operator, Renaissance Technologies, risk free rate, risk tolerance, risk-adjusted returns, risk/return, Robert Metcalfe, Ronald Reagan, Rubik’s Cube, Savings and loan crisis, semantic web, Sharpe ratio, short selling, short squeeze, Silicon Valley, Small Order Execution System, smart grid, smart meter, social web, South Sea Bubble, statistical arbitrage, statistical model, Steve Jobs, Steven Levy, stock buybacks, Tacoma Narrows Bridge, the scientific method, The Wisdom of Crowds, time value of money, tontine, too big to fail, transaction costs, Turing machine, two and twenty, Upton Sinclair, value at risk, value engineering, Vernor Vinge, Wayback Machine, yield curve, Yogi Berra, your tax dollars at work

Smart meters with Internet communication capabilities were the enabling technology for the next wave of technologydriven savings, designed to bring the suppliers into the process. This was 336 Nerds on Wall Str eet done using simple command-and-control load-shedding measures that allowed utilities, with prior agreement of larger customers, to shed loads during periods of peak demand or to remotely adjust airconditioning thermostats upward to reduce demand when needed. The next step, still in its infancy, is to introduce real-time electricity pricing. This will allow consumers to make their own economically motivated load-shedding decisions, and to program their meters to implement those decisions for them. Day Trading for Electrons In the best seller Hot, Flat, and Crowded: Why We Need a Green Revolution—and How It Can Renew America (Farrar, Straus & Giroux, 2008), New York Times columnist Thomas Friedman calls the use of smart meters to create a market-based system of energy technology (ET), “when IT meets ET—day trading for electrons.”


pages: 505 words: 147,916

Adventures in the Anthropocene: A Journey to the Heart of the Planet We Made by Gaia Vince

3D printing, agricultural Revolution, Anthropocene, bank run, biodiversity loss, car-free, carbon footprint, carbon tax, circular economy, citizen journalism, clean water, climate change refugee, congestion charging, crowdsourcing, decarbonisation, deindustrialization, driverless car, energy security, failed state, Google Earth, Haber-Bosch Process, hive mind, hobby farmer, informal economy, Intergovernmental Panel on Climate Change (IPCC), ITER tokamak, Kickstarter, Late Heavy Bombardment, load shedding, M-Pesa, Mars Rover, Masdar, megacity, megaproject, microdosing, mobile money, Neil Armstrong, ocean acidification, off grid, oil shale / tar sands, out of africa, Peter Thiel, phenotype, planetary scale, planned obsolescence, Ray Kurzweil, rewilding, Silicon Valley, Skype, smart cities, smart grid, smart meter, South China Sea, sovereign wealth fund, stem cell, supervolcano, sustainable-tourism, synthetic biology

It is hard to see how that will happen. Drought conditions are already causing blackouts as the nation’s hydropower struggles with lack of water and poor energy infrastructure. As throughout the developing world, most of the blackouts are planned load-shedding intervals, where the government tries to manage the poor supply by rationing electricity among different regions in turn. The load-shedding is seriously impacting Nepal’s fledgling electric car industry, its 700 Safa Tempos (three-wheeled electric passenger vans), which cleanly ply Kathmandu’s streets, charging at thirty-two stations and transporting around 100,000 people a day.


pages: 579 words: 164,339

Countdown: Our Last, Best Hope for a Future on Earth? by Alan Weisman

air freight, Albert Einstein, Anthropocene, anti-communist, Ayatollah Khomeini, Berlin Wall, biodiversity loss, Bretton Woods, British Empire, call centre, carbon footprint, clean water, colonial rule, David Attenborough, degrowth, demographic transition, Deng Xiaoping, Edward Jenner, El Camino Real, epigenetics, Filipino sailors, Garrett Hardin, Great Leap Forward, Haber-Bosch Process, happiness index / gross national happiness, haute couture, housing crisis, ice-free Arctic, Ignaz Semmelweis: hand washing, illegal immigration, immigration reform, Intergovernmental Panel on Climate Change (IPCC), land reform, liberation theology, load shedding, Louis Pasteur, Mahatma Gandhi, Mahbub ul Haq, megacity, Menlo Park, Michael Shellenberger, microdosing, Money creation, new economy, ocean acidification, oil shale / tar sands, out of africa, Pearl River Delta, planetary scale, Ponzi scheme, race to the bottom, rolling blackouts, Ronald Reagan, Satyajit Das, Seymour Hersh, Silicon Valley, South China Sea, stem cell, Stephen Hawking, Stewart Brand, unemployed young men, upwardly mobile, urban sprawl, Whole Earth Catalog, WikiLeaks

Everyone is back: men in white kurtis and prayer caps; hijab’d women brilliant as tropical birdlife, wrapped in multihued loose pants and tunics called salwar kameez; other women in black chadors, even transvestites in chadors, all threading their way through the stalled traffic, buying provisions and tea. Since 10:00 a.m., that traffic has gone from mere paralysis to pandemonium, as Lyari’s stoplights are out for the next three hours. All but the most privileged parts of Karachi are subject to load-shedding—daily rolling blackouts—because the city can’t possibly keep up with demand. There were fewer than a half-million people here in 1947. Today’s 21 million is a forty-two-fold increase. No one could have prepared for this. Three days earlier, when the grenade attacks began, everyone stayed hidden until long after the explosions ended.

He has six children with his first wife, and three with her. She wants to keep pleasing him so he’ll be attentive. Alma takes her hand. “But in your condition, more pregnancies are a risk.” The overhead fan quits and the deadened air is immediately stifling. The women exchange worried glances, because load-shedding was over for the day, and unexpected blackouts often signal yet a new civil disturbance. Asma pulls a flashlight from her desk drawer and motions for the next client. An entire bench stands and approaches: five women in white burqas bordered with lace, their dark eyes peering through woven lattice grills in their face hoods.


pages: 233 words: 69,745

The Reluctant Carer: Dispatches From the Edge of Life by The Reluctant Carer

call centre, cognitive dissonance, deskilling, disinformation, gig economy, Jeff Bezos, load shedding, place-making, stem cell, telemarketer, trolley problem

‘I couldn’t stop looking at it!’ says Mum of one’s nose-piercing. It’s as though characters from their television have walked directly into the room. 25 January 2018 I get an encouraging email from my friend Ron who lives abroad and knows a lot about energy: In electricity there is a term: load-shedding. Required to keep the power supply constant, in some way. A technical thing. I hope for some equivalent for you. He is right. I go to London, shed that load a little, write and reflect and confess. The Mighty Fall 28 January 2018 In my brief absence my benign, or perhaps lacklustre, regime has been replaced by a surveillance-gathering operation run by the unlikely but effective partnership of my deaf mum and suspicious sister.


Engineering Security by Peter Gutmann

active measures, address space layout randomization, air gap, algorithmic trading, Amazon Web Services, Asperger Syndrome, bank run, barriers to entry, bitcoin, Brian Krebs, business process, call centre, card file, cloud computing, cognitive bias, cognitive dissonance, cognitive load, combinatorial explosion, Credit Default Swap, crowdsourcing, cryptocurrency, Daniel Kahneman / Amos Tversky, Debian, domain-specific language, Donald Davies, Donald Knuth, double helix, Dr. Strangelove, Dunning–Kruger effect, en.wikipedia.org, endowment effect, false flag, fault tolerance, Firefox, fundamental attribution error, George Akerlof, glass ceiling, GnuPG, Google Chrome, Hacker News, information security, iterative process, Jacob Appelbaum, Jane Jacobs, Jeff Bezos, John Conway, John Gilmore, John Markoff, John von Neumann, Ken Thompson, Kickstarter, lake wobegon effect, Laplace demon, linear programming, litecoin, load shedding, MITM: man-in-the-middle, Multics, Network effects, nocebo, operational security, Paradox of Choice, Parkinson's law, pattern recognition, peer-to-peer, Pierre-Simon Laplace, place-making, post-materialism, QR code, quantum cryptography, race to the bottom, random walk, recommendation engine, RFID, risk tolerance, Robert Metcalfe, rolling blackouts, Ruby on Rails, Sapir-Whorf hypothesis, Satoshi Nakamoto, security theater, semantic web, seminal paper, Skype, slashdot, smart meter, social intelligence, speech recognition, SQL injection, statistical model, Steve Jobs, Steven Pinker, Stuxnet, sunk-cost fallacy, supply-chain attack, telemarketer, text mining, the built environment, The Death and Life of Great American Cities, The Market for Lemons, the payments system, Therac-25, too big to fail, Tragedy of the Commons, Turing complete, Turing machine, Turing test, Wayback Machine, web application, web of trust, x509 certificate, Y2K, zero day, Zimmermann PGP

A novice driver has to manually and consciously perform actions such as changing gears and checking the rear-view mirror, while for an experienced driver these actions occur automatically without any conscious effort. To cope with the inability to handle the driving process via automatic actions, novice drivers load-shed by ignoring one of the two main aspects of driving (speed control and steering), with the result that they crawl down the road at an irritatingly slow speed while they concentrate on steering. It’s not until both aspects of vehicle control have become automatic processes that the novices progress to the level of more experienced drivers [108]. 138 Psychology This load-shedding phenomenon is particularly pernicious in situations like aircraft control, where a multitude of events such as an unexpected occurrence during an already complex operation like takeoff or landing can overload a pilot’s ability to maintain full awareness of the situation [109][110][111].

It’s not until both aspects of vehicle control have become automatic processes that the novices progress to the level of more experienced drivers [108]. 138 Psychology This load-shedding phenomenon is particularly pernicious in situations like aircraft control, where a multitude of events such as an unexpected occurrence during an already complex operation like takeoff or landing can overload a pilot’s ability to maintain full awareness of the situation [109][110][111]. In order to cope with the situational awareness problem the pilot load-sheds, and the result of this process is written up by the board of inquiry as “pilot error”. Mind you, you can sometimes use the load-shedding phenomenon to your advantage. If you suspect that someone’s lying to you, you can increase their cognitive load to the point where they can no longer both maintain the lie and deal with the extra load. One way of doing this is to get them to narrate their story in reverse, which is awkward because it runs counter to the natural sequence of events and increases the chances that the liar will trip themselves up [112].

You can see the conflict between conscious and unconscious processing yourself if you write something simple like your name or the weekday repeatedly across a piece of paper and at some point start counting backwards from 100 while you write. Look at what happens to either your writing speed or writing quality when you do this, depending on which load-shedding strategy you choose to adopt. Now try it again but this time sign your name (an automatic process for most people) and see what happens. This simple experiment in fact mirrors some of the early investigations into the phenomenon of attention that were carried out in the 1950s, which involved seeing whether and how much performing one task interfered with another [115][116].


pages: 247 words: 78,961

The Return of Marco Polo's World: War, Strategy, and American Interests in the Twenty-First Century by Robert D. Kaplan

"World Economic Forum" Davos, Admiral Zheng, always be closing, California gold rush, collective bargaining, Deng Xiaoping, Donald Trump, Dr. Strangelove, failed state, Francis Fukuyama: the end of history, friendly fire, Great Leap Forward, Haight Ashbury, high-speed rail, kremlinology, load shedding, mass immigration, megacity, military-industrial complex, no-fly zone, oil-for-food scandal, one-China policy, Parag Khanna, Pax Mongolica, Ronald Reagan, South China Sea, sovereign wealth fund, the long tail, trade route, Westphalian system, Yom Kippur War

While it is true that Pakistan’s government is increasingly being held accountable by a burgeoning media and nongovernmental organizations—thereby expanding civil society at the top end of the spectrum in Islamabad and Lahore—and it is also true that interparty warfare in Islamabad has lessened somewhat, the country in fundamental ways continues to deteriorate. Electricity blackouts (“load shedding,” as they are called) are more persistent now than ever, and water shortages are worsening. The situation is fluid. Nuclear power and coal imports may soon alleviate the power blackouts, even as the army has reportedly moved away somewhat from encouraging Islamic radicalism. But Pakistan’s population growth is still above 2 percent annually, meaning its population doubles every thirty-five years.


pages: 290 words: 94,859

Wiseguy: The 25th Anniversary Edition by Nicholas Pileggi

air freight, half of the world's population has never made a phone call, index card, load shedding

Judy, who was a friend of the family, was already at my house when my brother and I got there. She looked like a Kansas preacher’s daughter. That, of course, was what made her such a good courier. Skinny, dirty-blond hair, dumb pink-and-blue hat and crummy Dacron clothes out of the Sears catalogue. Sometimes, with heavy loads, she’d borrow a baby for the trip. She looked so pathetic that the only people who ever stopped her were Travelers Aid social workers looking to stir up business. Judy was going to hang around the house until I got back with the stuff. Then, after we had all had dinner, I was going to drive her to the airport for her flight to Pittsburgh.


pages: 302 words: 92,206

Nomad Century: How Climate Migration Will Reshape Our World by Gaia Vince

3D printing, An Inconvenient Truth, Anthropocene, biodiversity loss, bitcoin, Boris Johnson, carbon tax, charter city, circular economy, clean water, colonial exploitation, coronavirus, COVID-19, decarbonisation, degrowth, Donald Trump, Dunbar number, European colonialism, failed state, gentrification, global pandemic, Global Witness, green new deal, Haber-Bosch Process, high-speed rail, housing crisis, ice-free Arctic, illegal immigration, Intergovernmental Panel on Climate Change (IPCC), invention of agriculture, invention of the printing press, job automation, joint-stock company, Kim Stanley Robinson, labour mobility, load shedding, lockdown, low skilled workers, Mahatma Gandhi, Malacca Straits, mass immigration, mass incarceration, mega-rich, megacity, negative emissions, new economy, ocean acidification, old age dependency ratio, open borders, Patri Friedman, Peace of Westphalia, Pearl River Delta, Peter Thiel, place-making, planetary scale, plyscraper, polynesian navigation, quantitative easing, randomized controlled trial, rewilding, Rishi Sunak, sharing economy, Shenzhen special economic zone , Silicon Valley, special economic zone, trade route, transatlantic slave trade, undersea cable, urban planning, urban sprawl, white flight, women in the workforce, working-age population, zero-sum game, Zipcar

Demand for cooling will soar this century, becoming a key social justice issue, especially during heatwaves, when lack of access will prove deadly. Cooling already uses 20 per cent of global energy production, and this is expected to triple by 2050. Months of heatwave in the spring of 2022 across India and Pakistan meant hundreds of thousands of people were unable to work after 10 a.m., with load-shedding power outages leaving people without access to cooling or refrigeration. Cooling is not just going to be a problem in the tropics, where there is already fast-rising demand, but in today’s temperate zones where vast populations will be headed. Insulation will help manage this burden, and strategic use of water – used for cooling by architects and planners for centuries – will also play a role.


pages: 1,203 words: 124,556

Lonely Planet Cape Town & the Garden Route (Travel Guide) by Lucy Corne

Berlin Wall, British Empire, Cape to Cairo, carbon footprint, Day of the Dead, gentrification, haute couture, haute cuisine, load shedding, Mark Shuttleworth, mass immigration, Nelson Mandela, New Urbanism, retail therapy, Robert Gordon, Suez canal 1869, tech billionaire, upwardly mobile, urban renewal, urban sprawl

The Natural Environment Shaped over millions of years, the Cape’s dramatic features are carpeted by the richest floral kingdom in the world. Wine The forces that created this 350-year-old industry, and how it is tackling the working practices that blighted its past. Cape Town Today The legacy of the city’s stint as World Design Capital 2014 (WDC2014); the threat of electricity blackouts (known as ‘load shedding’) due to a struggling national grid; political tussles between the Democratic Alliance (DA), who run the city and the Western Cape, and national governing party the African National Congress (ANC); and is Cape Town the most racist city in South Africa? These are among the ongoing conversation topics across the Mother City.


pages: 689 words: 134,457

When McKinsey Comes to Town: The Hidden Influence of the World's Most Powerful Consulting Firm by Walt Bogdanich, Michael Forsythe

"RICO laws" OR "Racketeer Influenced and Corrupt Organizations", "World Economic Forum" Davos, activist fund / activist shareholder / activist investor, Affordable Care Act / Obamacare, Alistair Cooke, Amazon Web Services, An Inconvenient Truth, asset light, asset-backed security, Atul Gawande, Bear Stearns, Boris Johnson, British Empire, call centre, Cambridge Analytica, carbon footprint, Citizen Lab, cognitive dissonance, collective bargaining, compensation consultant, coronavirus, corporate governance, corporate social responsibility, Corrections Corporation of America, COVID-19, creative destruction, Credit Default Swap, crony capitalism, data science, David Attenborough, decarbonisation, deindustrialization, disinformation, disruptive innovation, do well by doing good, don't be evil, Donald Trump, double entry bookkeeping, facts on the ground, failed state, financial engineering, full employment, future of work, George Floyd, Gini coefficient, Glass-Steagall Act, global pandemic, illegal immigration, income inequality, information security, interchangeable parts, Intergovernmental Panel on Climate Change (IPCC), invisible hand, job satisfaction, job-hopping, junk bonds, Kenneth Arrow, Kickstarter, load shedding, Mark Zuckerberg, megaproject, Moneyball by Michael Lewis explains big data, mortgage debt, Multics, Nelson Mandela, obamacare, offshore financial centre, old-boy network, opioid epidemic / opioid crisis, profit maximization, public intellectual, RAND corporation, Rutger Bregman, scientific management, sentiment analysis, shareholder value, Sheryl Sandberg, Silicon Valley, smart cities, smart meter, South China Sea, sovereign wealth fund, tech worker, The future is already here, The Nature of the Firm, too big to fail, urban planning, WikiLeaks, working poor, Yogi Berra, zero-sum game

GO TO NOTE REFERENCE IN TEXT “For almost 95 years”: “McKinsey’s Partners Suffer from Collective Self-Delusion,” Economist, March 4, 2021. GO TO NOTE REFERENCE IN TEXT “When you have people”: Testimony of Mieszala to Commission of Inquiry into State Capture, Dec. 10, 2020, 157. GO TO NOTE REFERENCE IN TEXT In January 2020, the company was forced: “Eskom Misled Ramaphosa About Extent of Load-Shedding: David Mabuza,” TimesLIVE, Jan. 9, 2020. GO TO NOTE REFERENCE IN TEXT South Africa’s president had cut short: “President Cuts Short Egyptian Visit to Attend to Electricity Crisis,” South African Government News Agency, Dec. 11, 2019. GO TO NOTE REFERENCE IN TEXT The media reported that 2021 could: Jackie Cameron, “Loadshedding: 2021 to Be Worst Year Yet for SA Electricity Crisis—Chris Yelland, Energy Expert,” BizNews, Jan. 20, 2021.


pages: 741 words: 179,454

Extreme Money: Masters of the Universe and the Cult of Risk by Satyajit Das

"RICO laws" OR "Racketeer Influenced and Corrupt Organizations", "there is no alternative" (TINA), "World Economic Forum" Davos, affirmative action, Alan Greenspan, Albert Einstein, algorithmic trading, Andy Kessler, AOL-Time Warner, Asian financial crisis, asset allocation, asset-backed security, bank run, banking crisis, banks create money, Basel III, Bear Stearns, behavioural economics, Benoit Mandelbrot, Berlin Wall, Bernie Madoff, Big bang: deregulation of the City of London, Black Swan, Bonfire of the Vanities, bonus culture, book value, Bretton Woods, BRICs, British Empire, business cycle, buy the rumour, sell the news, capital asset pricing model, carbon credits, Carl Icahn, Carmen Reinhart, carried interest, Celtic Tiger, clean water, cognitive dissonance, collapse of Lehman Brothers, collateralized debt obligation, corporate governance, corporate raider, creative destruction, credit crunch, Credit Default Swap, credit default swaps / collateralized debt obligations, currency risk, Daniel Kahneman / Amos Tversky, deal flow, debt deflation, Deng Xiaoping, deskilling, discrete time, diversification, diversified portfolio, Doomsday Clock, Dr. Strangelove, Dutch auction, Edward Thorp, Emanuel Derman, en.wikipedia.org, Eugene Fama: efficient market hypothesis, eurozone crisis, Everybody Ought to Be Rich, Fall of the Berlin Wall, financial engineering, financial independence, financial innovation, financial thriller, fixed income, foreign exchange controls, full employment, Glass-Steagall Act, global reserve currency, Goldman Sachs: Vampire Squid, Goodhart's law, Gordon Gekko, greed is good, Greenspan put, happiness index / gross national happiness, haute cuisine, Herman Kahn, high net worth, Hyman Minsky, index fund, information asymmetry, interest rate swap, invention of the wheel, invisible hand, Isaac Newton, James Carville said: "I would like to be reincarnated as the bond market. You can intimidate everybody.", job automation, Johann Wolfgang von Goethe, John Bogle, John Meriwether, joint-stock company, Jones Act, Joseph Schumpeter, junk bonds, Kenneth Arrow, Kenneth Rogoff, Kevin Kelly, laissez-faire capitalism, load shedding, locking in a profit, Long Term Capital Management, Louis Bachelier, low interest rates, margin call, market bubble, market fundamentalism, Market Wizards by Jack D. Schwager, Marshall McLuhan, Martin Wolf, mega-rich, merger arbitrage, Michael Milken, Mikhail Gorbachev, Milgram experiment, military-industrial complex, Minsky moment, money market fund, Mont Pelerin Society, moral hazard, mortgage debt, mortgage tax deduction, mutually assured destruction, Myron Scholes, Naomi Klein, National Debt Clock, negative equity, NetJets, Network effects, new economy, Nick Leeson, Nixon shock, Northern Rock, nuclear winter, oil shock, Own Your Own Home, Paul Samuelson, pets.com, Philip Mirowski, Phillips curve, planned obsolescence, plutocrats, Ponzi scheme, price anchoring, price stability, profit maximization, proprietary trading, public intellectual, quantitative easing, quantitative trading / quantitative finance, Ralph Nader, RAND corporation, random walk, Ray Kurzweil, regulatory arbitrage, Reminiscences of a Stock Operator, rent control, rent-seeking, reserve currency, Richard Feynman, Richard Thaler, Right to Buy, risk free rate, risk-adjusted returns, risk/return, road to serfdom, Robert Shiller, Rod Stewart played at Stephen Schwarzman birthday party, rolodex, Ronald Reagan, Ronald Reagan: Tear down this wall, Satyajit Das, savings glut, shareholder value, Sharpe ratio, short selling, short squeeze, Silicon Valley, six sigma, Slavoj Žižek, South Sea Bubble, special economic zone, statistical model, Stephen Hawking, Steve Jobs, stock buybacks, survivorship bias, tail risk, Teledyne, The Chicago School, The Great Moderation, the market place, the medium is the message, The Myth of the Rational Market, The Nature of the Firm, the new new thing, The Predators' Ball, The Theory of the Leisure Class by Thorstein Veblen, The Wealth of Nations by Adam Smith, Thorstein Veblen, too big to fail, trickle-down economics, Turing test, two and twenty, Upton Sinclair, value at risk, Yogi Berra, zero-coupon bond, zero-sum game

A minister extols the virtues of India, citing statistics on growth, resource availability, and opportunities. There is no mention of the fact that the vast majority of the Indian population has no access to sanitation, clean water, education, or healthcare. There is no mention of the aging colonial era infrastructure where inadequate electricity supply results in daily load shedding or brownouts interrupting power supplies for several hours most days. An American banker finds India fascinating and full of opportunity. “A billion people, a billion consumers, wow!” He is fascinated that I, an Indian, do not speak Indian but Bengali, one of the hundreds of languages spoken in India.


pages: 834 words: 180,700

The Architecture of Open Source Applications by Amy Brown, Greg Wilson

8-hour work day, anti-pattern, bioinformatics, business logic, c2.com, cloud computing, cognitive load, collaborative editing, combinatorial explosion, computer vision, continuous integration, Conway's law, create, read, update, delete, David Heinemeier Hansson, Debian, domain-specific language, Donald Knuth, en.wikipedia.org, fault tolerance, finite state, Firefox, Free Software Foundation, friendly fire, functional programming, Guido van Rossum, Ken Thompson, linked data, load shedding, locality of reference, loose coupling, Mars Rover, MITM: man-in-the-middle, MVC pattern, One Laptop per Child (OLPC), peer-to-peer, Perl 6, premature optimization, recommendation engine, revision control, Ruby on Rails, side project, Skype, slashdot, social web, speech recognition, the scientific method, The Wisdom of Crowds, web application, WebSocket

Unlike the consistent hashing approach, two keys that are next to each other in the key's sort order are likely to appear in the same partition. This reduces the size of the routing metadata, as large ranges are compressed to [start, end] markers. In adding active record-keeping of the range-to-server mapping, the range partitioning approach allows for more fine-grained control of load-shedding from heavily loaded servers. If a specific key range sees higher traffic than other ranges, a load manager can reduce the size of the range on that server, or reduce the number of shards that this server serves. The added freedom to actively manage load comes at the expense of extra architectural components which monitor and route shards.