[AusNOG] census issues tonight

Wed Aug 10 09:11:01 EST 2016

So.what evidence (if any) is there to suggest that this was a DDoS attack
originating from O/S.?

I'm assuming that ASD are onto it.and that they are aware of proxy
chaining.?

Was Tomcat properly hardened, was there any form of DDoS mitigation in
place?

One would think that, given the current animosity between Australia and
China (Olympics and Spratleys).

Not only would load testing have been in place, but a comprehensive PenTest
would have been conducted.?

Andy Taylor

Technical Director

0424 656 973

 <http://www.coastalaudio.com.au/> www.coastalaudio.com.au 

From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Nathan
Brookfield
Sent: Wednesday, 10 August 2016 8:35 AM
To: James Hodgkinson <yaleman at ricetek.net>; James Braunegg
<james.braunegg at micron21.com>; Tim Raphael <raphael.timothy at gmail.com>;
ausnog at lists.ausnog.net
Subject: Re: [AusNOG] census issues tonight

Didn't have to do with Geolocation at all, just don't advertise the prefix
down any international paths.  If your not on a domestic carrier, no site,
that simple.

I'm sure there is a small percentage that would have had issues who use
Proxies, VPN's etc but would be a small minority compared to the amount of
people who were inconvenienced.

Kindest Regards,

Nathan Brookfield (VK2NAB)

Chief Executive Officer

Simtronic Technologies Pty Ltd

Local: (02) 4749 4949 | Fax: (02) 4749 4950 | Direct: (02) 4749 4951

Web:  <http://www.simtronic.com.au/> http://www.simtronic.com.au | E-mail:
<mailto:nathan.brookfield at simtronic.com.au>
nathan.brookfield at simtronic.com.au

  _____  

From: James Hodgkinson <yaleman at ricetek.net <mailto:yaleman at ricetek.net> >
Sent: Wednesday, 10 August 2016 8:11 AM
To: James Braunegg; Nathan Brookfield; Tim Raphael; ausnog at lists.ausnog.net
<mailto:ausnog at lists.ausnog.net> 
Subject: Re: [AusNOG] census issues tonight 

"I still don't understand why the /24 was even advertised internationally at
all.  doesn't make sense for a website that only Australians should have
been accessing from within Australia."

Because geolocation is poor guesswork at the best of times? We run into
issues with it all the time. I'm surprised they had their "CDN" nodes in the
same range (and certificate, seriously, lazy much?)

James

On Tue, 9 Aug 2016, at 23:44, James Braunegg wrote:

You don't reroute traffic.. From having an application fail.

You might reroute traffic from network congestion.. But you only black hole
/32 routes when you want to stop unwanted traffic with a sledge hammer..

I still don't understand why the /24 was even advertised internationally at
all.  doesn't make sense for a website that only Australians should have
been accessing from within Australia.

Kindest Regards

James Braunegg
P:  1300 769 972  | M:  0488 997 207 | D:  (03) 9751 7616

E:  <mailto:james.braunegg at micron21.com> james.braunegg at micron21.com  | ABN:
12 109 977 666 
W:   <http://www.micron21.com/ddos-protection>
www.micron21.com/ddos-protection T: @micron21

Follow us on  <http://www.twitter.com/micron21> Twitter for important
service and system updates.

This message is intended for the addressee named above. It may contain
privileged or confidential information. If you are not the intended
recipient of this message you must not use, copy, distribute or disclose it
to anyone other than the addressee. If you have received this message in
error please return the message to the sender by replying to it and then
delete the message from your computer.

From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Nathan
Brookfield
Sent: Tuesday, 9 August 2016 11:39 PM
To: Tim Raphael <raphael.timothy at gmail.com
<mailto:raphael.timothy at gmail.com> >; ausnog at lists.ausnog.net
<mailto:ausnog at lists.ausnog.net> 
Subject: Re: [AusNOG] census issues tonight

If you add that to Vocus blackholing some /32's intermittently, I think we
can safely assume that they were a tasty target  What an absolute
disaster.....

Kindest Regards,

Nathan Brookfield (VK2NAB)

Chief Executive Officer

Simtronic Technologies Pty Ltd

Local: (02) 4749 4949 | Fax: (02) 4749 4950 | Direct: (02) 4749 4951

Web:  <http://www.simtronic.com.au/> http://www.simtronic.com.au | E-mail:
<mailto:nathan.brookfield at simtronic.com.au>
nathan.brookfield at simtronic.com.au

  _____  

From: AusNOG <ausnog-bounces at lists.ausnog.net
<mailto:ausnog-bounces at lists.ausnog.net> > on behalf of Tim Raphael
<raphael.timothy at gmail.com <mailto:raphael.timothy at gmail.com> >
Sent: Tuesday, 9 August 2016 11:25 PM
To: ausnog at lists.ausnog.net <mailto:ausnog at lists.ausnog.net> 
Subject: Re: [AusNOG] census issues tonight

So what do we think?

DDoS? (as in illegitimate traffic)

I think the evidence might be there given we saw some Telstra route changes
that look an awful lot like scrubbing near the customer tail...

- Tim

On 9 Aug 2016, at 9:23 PM, Michael Schipp <michaelsc at mellanox.com
<mailto:michaelsc at mellanox.com> > wrote:

So they now pay me $180?

From: AusNOG [ <mailto:ausnog-bounces at lists.ausnog.net>
mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Damian Guppy
Sent: Tuesday, 9 August 2016 11:11 PM
To: Joshua D'Alton < <mailto:joshua at railgun.com.au> joshua at railgun.com.au>
Cc:  <mailto:ausnog at lists.ausnog.net> ausnog at lists.ausnog.net
Subject: Re: [AusNOG] census issues tonight

Well, looks like they have given up...

 <https://twitter.com/ABSCensus/status/762996836357419008>
https://twitter.com/ABSCensus/status/762996836357419008

"ABS & Census website are unavailable. The service won't be restored
tonight. We will update you in AM. We apologise for the inconvenience."

On Tue, Aug 9, 2016 at 9:05 PM, Joshua D'Alton <
<mailto:joshua at railgun.com.au> joshua at railgun.com.au> wrote:

Surely those 11 would be the load balancing stack in front, then again 24
million requests lasting 10 minutes over 72hours is only ~1k requests per
second, probably easily manageable by 11 servers with a bunch of big SQL
boxes behind it.

Tracing to the IPs gives different results.

.1 includes ibm.

.2 goes straight.

.3 includes ibm

.4 goes straight

.5 includes ibm but fails to reach target

.6... *

On Tue, Aug 9, 2016 at 10:50 PM, James Braunegg <
<mailto:james.braunegg at micron21.com> james.braunegg at micron21.com> wrote:

Total of 11 servers used for the application stack..

 <https://twitter.com/oliyoung/status/761028000821288961>
https://twitter.com/oliyoung/status/761028000821288961

 <https://pbs.twimg.com/media/Co-2565VMAAZeBq.jpg>
https://pbs.twimg.com/media/Co-2565VMAAZeBq.jpg

Thanks Tom M for the links

Kindest Regards

James Braunegg
P:   <tel:1300%20769%20972> 1300 769 972  |  M:   <tel:0488%20997%20207>
0488 997 207 |  D:   <tel:%2803%29%209751%207616> (03) 9751 7616

E:    <mailto:james.braunegg at micron21.com> james.braunegg at micron21.com  |
ABN:  12 109 977 666   
W:   <http://www.micron21.com/ddos-protection>
www.micron21.com/ddos-protection   T: @micron21

Follow us on  <http://www.twitter.com/micron21> Twitter for important
service and system updates.

<image002.jpg>

This message is intended for the addressee named above. It may contain
privileged or confidential information. If you are not the intended
recipient of this message you must not use, copy, distribute or disclose it
to anyone other than the addressee. If you have received this message in
error please return the message to the sender by replying to it and then
delete the message from your computer.

From: AusNOG [mailto: <mailto:ausnog-bounces at lists.ausnog.net>
ausnog-bounces at lists.ausnog.net] On Behalf Of Troy Cowin
Sent: Tuesday, 9 August 2016 10:48 PM
To: Colin Stubbs < <mailto:colin.stubbs at equatetechnologies.com.au>
colin.stubbs at equatetechnologies.com.au>;  <mailto:ausnog at lists.ausnog.net>
ausnog at lists.ausnog.net

Subject: Re: [AusNOG] census issues tonight

And the rest.

 <http://i.imgur.com/N7tlah4.png> http://i.imgur.com/N7tlah4.png

From: AusNOG [ <mailto:ausnog-bounces at lists.ausnog.net>
mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Colin Stubbs
Sent: Tuesday, 9 August 2016 8:35 PM
To: Glenn Powell < <mailto:glenn at glennbridge.com.au>
glenn at glennbridge.com.au>
Cc:  <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
Subject: Re: [AusNOG] census issues tonight

$54,367.50 to be precise...

<https://www.tenders.gov.au/?event=public.cn.view&CNUUID=069BE127-081D-5AF9-
EB096752848D31A8>
https://www.tenders.gov.au/?event=public.cn.view&CNUUID=069BE127-081D-5AF9-E
B096752848D31A8

On 9 Aug 2016 8:14 PM, "Glenn Powell" < <mailto:glenn at glennbridge.com.au>
glenn at glennbridge.com.au> wrote:

They spent $50k on a 3rd party to do load testing, I wonder what the success
criteria of that testing looked like?

On 9 Aug 2016, at 8:09 PM, Brent Paddon < <mailto:brent.paddon at gmail.com>
brent.paddon at gmail.com> wrote:

Only cost $9.6M to host the site... 

Brent

On Tue, Aug 9, 2016 at 8:07 PM, James Braunegg <
<mailto:james.braunegg at micron21.com> james.braunegg at micron21.com> wrote:

Dead as a doornail .. from Telstra.

How sad. Now What.

Kindest Regards

James Braunegg
P:   <tel:1300%20769%20972> 1300 769 972  |  M:   <tel:0488%20997%20207>
0488 997 207 |  D:   <tel:%2803%29%209751%207616> (03) 9751 7616

E:    <mailto:james.braunegg at micron21.com> james.braunegg at micron21.com  |
ABN:  12 109 977 666   
W:   <http://www.micron21.com/ddos-protection>
www.micron21.com/ddos-protection   T: @micron21

Follow us on  <http://www.twitter.com/micron21> Twitter for important
service and system updates.

<image001.jpg>

This message is intended for the addressee named above. It may contain
privileged or confidential information. If you are not the intended
recipient of this message you must not use, copy, distribute or disclose it
to anyone other than the addressee. If you have received this message in
error please return the message to the sender by replying to it and then
delete the message from your computer.

From: AusNOG [mailto: <mailto:ausnog-bounces at lists.ausnog.net>
ausnog-bounces at lists.ausnog.net] On Behalf Of John Lindsay
Sent: Tuesday, 9 August 2016 7:52 PM
To: Joseph Goldman < <mailto:joe at apcs.com.au> joe at apcs.com.au>
Cc:  <mailto:ausnog at lists.ausnog.net> ausnog at lists.ausnog.net
Subject: Re: [AusNOG] census issues tonight

It's turned to glue for me.

Was great an hour or so ago.

John Lindsay

 <mailto:johnslindsay at mac.com> johnslindsay at mac.com

 <tel:%2B61403577711> +61403577711

On 9 Aug 2016, at 6:51 PM, Joseph Goldman < <mailto:joe at apcs.com.au>
joe at apcs.com.au> wrote:

Just completed it then was super snappy for me, so assuming they either
ramped up servers or people gave up lol.

On 09/08/16 18:56, Joshua D'Alton wrote:

Their phone number too just rings out with "we're too busy, please call
back". Probably due to aforementioned issues.. and then the few million old
people who are trying to do it over the phone. Or people who need to call
because they were doing it and their session died and their 9 digit code no
longer works :/

On Tue, Aug 9, 2016 at 6:33 PM, Troy Cowin <
<mailto:troy at perthsystems.com.au> troy at perthsystems.com.au> wrote:

Yep lots of various errors even earlier this morning - took about 5 minutes
of refreshing to actually get into the site. Once in though it worked ok,
hate to see how it will handle the load now its peak hour.

From: AusNOG [mailto: <mailto:ausnog-bounces at lists.ausnog.net>
ausnog-bounces at lists.ausnog.net] On Behalf Of Ben Cornish
Sent: Tuesday, 9 August 2016 4:28 PM
To:  <mailto:ausnog at lists.ausnog.net> ausnog at lists.ausnog.net
Subject: [AusNOG] census issues tonight

Not surprising - but anyone else seeing issues with the Census site tonight
?

Seems Soft layer is serving up parts of this behind the scenes..

Im seeing  504 errors from soft layer for large chunks of IP ranges - but
not all.

Anyone else seeing this ?

Ben Cornish
Chief Operating Officer

Over the Wire Holdings Ltd (ASX:OTW)

GPO Box 1807 Brisbane, QLD, 4001, Australia

Level 1, 24 Little Edward Street, Spring Hill, QLD 4000
t     <tel:1300%20689%20689> 1300 689 689

m   <tel:0417%20617%20204> 0417 617 204

e    <mailto:Ben.Cornish at overthewire.com.au> Ben.Cornish at overthewire.com.au
<http://www.overthewire.com.au/> www.overthewire.com.au

_______________________________________________
AusNOG mailing list
 <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
 <http://lists.ausnog.net/mailman/listinfo/ausnog>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
 <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
 <http://lists.ausnog.net/mailman/listinfo/ausnog>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
 <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
 <http://lists.ausnog.net/mailman/listinfo/ausnog>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
 <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
 <http://lists.ausnog.net/mailman/listinfo/ausnog>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
 <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
 <http://lists.ausnog.net/mailman/listinfo/ausnog>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
 <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
 <http://lists.ausnog.net/mailman/listinfo/ausnog>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
 <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
 <http://lists.ausnog.net/mailman/listinfo/ausnog>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
 <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
 <http://lists.ausnog.net/mailman/listinfo/ausnog>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________
AusNOG mailing list
 <mailto:AusNOG at lists.ausnog.net> AusNOG at lists.ausnog.net
 <http://lists.ausnog.net/mailman/listinfo/ausnog>
http://lists.ausnog.net/mailman/listinfo/ausnog

_______________________________________________

AusNOG mailing list

AusNOG at lists.ausnog.net <mailto:AusNOG at lists.ausnog.net> 

http://lists.ausnog.net/mailman/listinfo/ausnog

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20160810/5a398fbc/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 16869 bytes
Desc: not available
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20160810/5a398fbc/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.jpg
Type: image/jpeg
Size: 2683 bytes
Desc: not available
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20160810/5a398fbc/attachment-0001.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image005.png
Type: image/png
Size: 506 bytes
Desc: not available
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20160810/5a398fbc/attachment-0003.png>