The new face of disaster

I remember a few years back, when I heard about the blackouts in California (oh yes, the good ol’ Enron days). It was quite shocking to hear that major dot-coms were down for hours. Even the “365 Main” facility in San Francisco with its earthquake proof infrastructure lost power, proving that no matter how equipped, no single location can withstand a big disaster.

Nowadays this is less and less a real issue – hurricanes and power failures are not an excuse to stop providing service: Amazon and google showed that you can reach close to 100% reliability (barring software bugs) by eliminating all physical single points of failure. Today in the cloud age, every web site service can get Amazon-like reliability without worrying about a power failure in its office in Mountain View or a natural disaster in its colocation farm – and all this for hundreds of dollars a month.

But as the local disaster problem is solved, there’s a new one that may shape the way we think of disaster recovery. got hit by a massive DDoS attack on its DNS servers. This attack will have many casualties – not just’s users who may have their web sites unavailable if they used’s DNS services but also all those hit by the collateral damage; we don’t yet have a technical information on how the attack was done, but a DDoS attack is typically logical and not geographical – if your site is somehow ‘logically’ connected to a site that is being attacked, you will be DDoS’ed and that won’t be nice. When blue security was DDoS a few years ago the attackers decided to take down Blue Security’s providers along with anything hosted there, in any of the provider’s geographical location.

A DDoS attacks the server wherever he is – if you span your server across multiple physical locations the attack will be done on all of them; there is always a limit to the number of transactions you can handle in a single second, and once the attacking botnet passes this limit your services will effectively be denied. You will then have nothing to do but lean back in your chair and wait for the attack to end, counting the lost visitors/revenue/reputation with every minute passing.

While the cloud can save you from Hurricane Katrina, if someone decides to DDoS they only need to pay a fee; there is nothing facebook – with its massive server infrastructure – can do to stop them. In fact, we don’t know of any real way to stop DDoS (snakeoil solutions aside) and Rob is very correct in saying that probably the only solution is raising security awareness to reduce the size of botnets and make DDoS less practical (or more expensive). Until that happens, I wonder who will be the first to use DDoS to take out a competitor?