[AusNOG] After Sandy Knocks Out Power, ... (huffingtonpost.com)

[Christopher Pollock]

A++ post, would agree again.

It's supremely easy to sit back and criticise people for not doing blank to
prevent an outage or datacentre failure.  You don't have to peruse
Whirlpool for long after any outage to hear people 'why didnt they do this
or that'. And in a perfect world, our HA plans would all go exactly to plan
and every scenario could be prepared for and anything in the world could
happen and everything would be fine anyway.

The reality of it is, if you've spent any real amount of time building &
managing datacentres or maintaining highly-available services, you know
that a HA plan is really only as good as the circumstances you can feasibly
spend money preparing for.  Sometimes it means running to Mitre 10 to buy
their entire stock of fans.  Other times it means carrying hundreds of
litres of diesel up a dark staircase.

There is basically no amount of money you can spend that guarantees your
stuff won't fall over, so my hat is off to the people who got hit by a bad
situation and put some hard work into staying online.

--
Christopher Pollock,
io Networks Pty Ltd.
e. chris at ionetworks.com.au
p. 1300 1 2 4 8 16
d. 07 3188 7588
m. 0410 747 765
skype: christopherpollock
twitter.com/chrisionetworks
http://www.ionetworks.com.au
In-house, Outsourced.



On Fri, Nov 2, 2012 at 7:01 AM, Mark Newton <newton at atdot.dotat.org> wrote:

> That's all very well and good, but it seems to me that they've just
> suffered a 100 year storm and they stayed up.
>
> I'm sure you can armchair quarterback 'til the cows come home, but you're
> basically criticizing a success.
>
> On 02/11/2012, at 6:35 AM, Martin Hepworth <maxsec at gmail.com> wrote:
>
> That still leaves them with a single data centre, your DR should be in a
> separate physical location at least 45 miles away
>
>
> ... if you're a bank.  Which they aren't.
>
> Note also that you're talking about DR, but what they've actually
> demonstrated is HA.  The article already said that they had a DR plan to
> relocate services to another datacentre; they didn't need to invoke it
> because they didn't have a disaster, *because they stayed up.*
> *
> *
> If they actually went 100% down, then brought themselves up at another
> datacentre 6 hours later, you'd be praising them for having a well thought
> out old-school DR plan, right?
>
> (some enterprises have DR plans which take anything up to a week to
> execute.  DR != HA.)
>
> This goes back to old school infosec on risk and costs to business of
> outages.
>
>
> Yes, and also the cost of infrastructure.  One doesn't protect one's fruit
> bowl with a $50,000 safe.
>
> Problem with alot of the new facilities being build on Saas/cloudy
> offerings is that theyve grown so fast theyve nit done some of thr basics
> and rely on luck to get out of problems!
>
>
> The problem that SaaS/cloud offerings have is that they're reliant on a
> software substrate that's rarely been tested in true adversity, and
> therefore rely quite a bit on trust.
>
> When the chips are down, is your cloud provider as good as they say they
> are?
>
>   - mark
>
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20121102/b4bd2c19/attachment.html>