[AusNOG] census issues tonight

[Alan Maher]

 From a Kiwi user perspective, this is familiar.

The NZ census was delayed due to the effects of the Christchurch 
earthquake taking out the infrastructure to a large city, and a sizeable 
percentage of the population.  When it was rescheduled, a couple of 
years ago, the online option was offered.  As an IT bloke with some 
experience of Govt. attempts at this stuff, I thought yup, this might 
prove interesting. And it was. My personal attempt to login using my ID 
number, personally sent to me by snail mail didn't work, due to network 
congestion at around 7.30pm. So, I thought it might work later, when the 
kids have gone to bed. At 9.30pm the user ID failed, due to it having 
been used before !?! hellooooo. So, I phoned the freephone number, 
explained the problem, and was given a new ID number to use. That didn't 
work either, as it timed out during the waiting process. After a third 
attempt, which repeated the same scenario, I gave up, ticked all the 
paper forms, and sent it back in the pre-paid envelope provided.

Govt. interaction with its citizens requires a fairly robust and secure 
system to be in place. It would appear that we share the same idiots in 
charge, and the same contractors promising the earth, and charging the moon.

Teacher's note: Could do better, if he/she actually did the homework.



On 10/08/2016 4:28 p.m., Mark Delany wrote:
> On 10Aug16, James Braunegg allegedly wrote:
>> No need for Geo Blocking.. that???s hard work
>>
>> Just only advertise the route locally within Australia i.e... to Optus, Telstra and on peering exchanges... Job done..
> Nope. Job not done. This sort of single-bullet approach is probably
> why they failed.
>
> If you want scale and resiliency there are many many things you do to
> ensure success. For example how would an AU-only route announcement
> protect against a DDOS initiated here? Australians love their ancient
> Windows boxen so there are plenty of locally available bots for rent.
>
> It's hard to know where to even begin with the census site as they got
> it wrong in so many ways. It's obvious they never even did a mental
> walk thru of what-ifs.
>
> Based on HTTP responses with failure text, we can guess that that they
> had a coupled system when a de-coupled one would have been more
> resilient. They relied on physical scaling which is obviously
> impossible to augment in any reasonable time frame. They did not do a
> trial run of anything to try and get a sense of the traffic profile so
> they were completely guessing. Why not get everyone to register a week
> beforehand to get a feel for the traffic and load? Their servers were
> centralized, which is an obvous no-no. Even their DNS setup was such
> that they couldn't swing traffic quickly if they had to.
>
> Their efforts at switching routing during the evening suggests that
> they though it was some sort of traffic based DOS, but as other
> observed, there is not a lot of evidence that that was actually the
> case. It looks like all they knew was that their service was failing
> and they were scrambling to deal with it. Did they do a practise run
> with an actually DDOS? Their 6h DNS TTL suggests not as that's one of
> the first things you want to be able to change rapidly.
>
> I also saw no evidence of their ability to gracefully degrade. Either
> they were up or they were down. No ability to redistribute the
> traffic, nor to have the browser-based JS reach for an alternative
> site or for the site to do less work when it got too busy, such as
> dump and defer validation.
>
> Their one bullet seems to be to have provisioned twice as much
> front-end server capacity as they thought they'd need. A mere 2x
> margin for a completely new, unknown traffic profile system? That's
> pretty scandalous for such a high-profile site right there.
>
>
> Mark.
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus