The sneeze that gave half the internet flu: Why the AWS outage was so costly

The Amazon Web Services outage on Monday proved that, even for platforms hosted by one of the world's biggest technology companies, things can go wrong quickly.

The domino effect of the debacle, which lasted for about 15 hours, disrupted several AWS-hosted services and platforms and was reminiscent of the faulty CrowdStrike update that brought down Microsoft Windows systems in July last year.

This time, it involved several of the world's most popular applications, from critical tech services including the cloud and cyber security to consumer staples such as payments.

Most AWS services back online after major disruption to global web platforms

Read More

“When AWS sneezes, half the internet catches the flu. Outages like this cause frustrated users, but also triggers a domino effect across payment flows," said Monica Eaton, chief executive of US payment services company Chargebacks911.

The services affected include failed authorisations, duplicate charges and broken confirmation pages – "all of that fuels a wave of disputes that merchants will be cleaning up for weeks", she added.

Net dependent

AWS is one of the world's biggest web-hosting providers with more than 76 million websites, storing location information, data and contact details, the tracker Built With said. That is a good chunk of the internet.

But, when an outage of this magnitude occurs, "the ripple effects can quickly spread across industries and into people's daily lives", said Rob van Lubek, vice president for Europe, the Middle East and Africa at US software development firm Dynatrace.

Such a global incident is "a clear reminder of how dependent our world has become on software and digital systems operating as expected", he added.

And this is not a mere technical glitch – it is a problem that costs businesses money. "The headlines will focus on streaming services being down. The real, untold story is the unrecoverable loss of conversions for millions of small businesses," said Ismael Wrixen, chief executive of New York-based software developer ThriveCart. "Every minute this occurs, entrepreneurs are learning the most painful lesson in e-commerce: your perfectly optimised ad funnel means nothing if the 'buy' button is dead."

That would cause financial losses for businesses, who should now brace themselves for frustrated consumers and the longer-term effects of an outage.

“What I expect now is a spike in ‘I never got my service’ or ‘I was charged twice’ claims. Many of those won’t be fraud, just confusion," Ms Eaton said. "But confusion is the number one driver of chargebacks. If merchants sit back and wait for disputes to roll in, they will bleed revenue unnecessarily.”

However, shares of AWS parent Amazon ended higher on Monday, as investors "were reminded how integral AWS is to global business", said Ipek Ozkardeskaya, a senior analyst at Swissquote Bank.

Out-of-office issue

AWS initially said the crisis, which turned out to be a problem with a domain name service, took root in its "US-East-1" region, which quickly spread across America and, eventually, other parts of the world. A total of 142 AWS services, including the popular Fortnite game, were affected, its final update found. Seattle-based AWS took about 15 hours to resolve the issue fully.

It is unclear how much that region contributes to AWS's operations, but the outage shows the intricacies of the internet, built for reliability, can also be its downfall. "The outage isn't just an 'East Coast AWS' problem, it's a reminder that 100 per cent uptime is a myth for everyone as the internet runs on shared infrastructure," Mr Wrixen said.

It also exposed the reality that many companies do not have contingencies for such an outage, which was only made worse because it happened outside regular working hours.

"The real story isn't just that AWS had a critical issue, but how many businesses discovered their platform partner had no plan for it, especially outside of US hours," Mr Wrixen said. "This is a harsh wake-up call about the critical need for multi-regional redundancy and intelligent architecture."

While large enterprises typically have systems in place to mitigate disruptions, it is not a one-size-fits-all solution. As with cyber attacks, anticipation and contingencies are key.

"The difference between disruption and recovery often comes down to visibility and speed – how fast an organisation can pinpoint what's gone wrong, understand why and act to restore service continuity," Mr van Lubek said. "The organisations best prepared for the future will be those that can see across their entire environment, anticipate risks and adapt quickly when the unexpected happens."