Application Security , Incident & Breach Response , Managed Detection & Response (MDR)

Widespread Internet Outages: Fastly Reports Problem Fixed

Access to Amazon, Google, Twitter and Others Disrupted by Content Delivery Network
Widespread Internet Outages: Fastly Reports Problem Fixed
The Down Detector website reported widespread outages, tracing to Fastly, beginning about 11 a.m. U.K. time (6 a.m. U.S. Eastern time).

Amazon, Google, Spotify and Twitter were among the sites that were unreachable Tuesday morning due to a configuration error at widely used content delivery network Fastly.

See Also: Panel Discussion | The 2021 Global State of Security

The website Down Detector, which tracks when internet services and apps are unreachable, reported widespread outages on Tuesday. Other unreachable sites included Amazon Web Services, CNN, Discord, HBO Max (part of AT&T's WarnerMedia), The New York Times, Reddit and the U.K. government's public-facing website, among others.

The outage began Tuesday morning and was quickly traced to Fastly, based in San Francisco. After about an hour, services began to get restored.

Fastly says the disruption involved a configuration error in its points of presence. Such CDNs - aka "edge clouds" - are meant to provide more rapid access to widely used sites by replicating the sites' content across points of presence around the world, which are ideally located as physically close to website users as possible.

But if a CDN itself goes down, it can leave numerous sites unreachable.

"We identified a service configuration that triggered disruptions across our POPs globally and have disabled that configuration," a Fastly spokeswoman tells Information Security Media Group. "Our global network is coming back online."

The company's website lists the current status of its network.

Fastly says the outage began about 6 a.m. U.S. East Coast time. At 6:36 a.m., it reported: "Fastly has observed recovery of all services and has resolved this incident."

Two hours later, the company said that all services had been restored, and the incident had been resolved, although "customers could continue to experience a period of increased origin load and lower cache hit ratio." Origin load refers to the load placed on customers' servers, while cache ratio measures how effective a cache is at meeting requests for content.

The disruptions due to the Fastly configuration error did not disrupt access to affected sites from all geographies. For example, some users in Berlin reported no outages, while outages were reported in such locations as London, Texas and New Zealand, The Guardian reports.

This isn't the first outage involving a CDN that knocked downstream customers offline.

On Aug. 30, 2020, for example, CenturyLink was offline for most of that Sunday morning, which in turn led to sites such as Cloudflare, Discord, Feedly, Hulu, PlayStation Network, Xbox Live and many others being unreachable.

The outage for Cloudflare, which is a CDN, in turn led to outages for dozens of its customers as well.

That followed a July 17, 2020, outage at Cloudflare that lasted a half hour, which the company blamed on a configuration error.

"The outage occurred because, while working on an unrelated issue with a segment of the backbone from Newark to Chicago, our network engineering team updated the configuration on a router in Atlanta to alleviate congestion," Cloudflare said at the time in a blog post. "This configuration contained an error that caused all traffic across our backbone to be sent to Atlanta. This quickly overwhelmed the Atlanta router and caused Cloudflare network locations connected to the backbone to fail."

About the Author

Mathew J. Schwartz

Mathew J. Schwartz

Executive Editor, DataBreachToday & Europe, ISMG

Schwartz is an award-winning journalist with two decades of experience in magazines, newspapers and electronic media. He has covered the information security and privacy sector throughout his career. Before joining Information Security Media Group in 2014, where he now serves as the executive editor, DataBreachToday and for European news coverage, Schwartz was the information security beat reporter for InformationWeek and a frequent contributor to DarkReading, among other publications. He lives in Scotland.

Around the Network

Our website uses cookies. Cookies enable us to provide the best experience possible and help us understand how visitors use our website. By browsing, you agree to our use of cookies.