[AusNOG] So who broke the internet now ?

[James Braunegg]

Found the source of the issue last night... Next Gen were doing some upgrades... which went a little wrong...

Dear Customer,

Please be advised the following Emergency Planned Outage Works by Nextgen Network has been scheduled.

This work is expected to result in an outage to your service/s listed below.

Start Date:       16/03/10            Start Time:      00:01 (AEDT)

Finish Date:     16/03/10            Finish Time:    02:00 (AEDT)

Description of Works:

Nextgen has organized an emergency planned outage to replace the faulty card on one of the core device.

Your service listed above will experience an outage of 30 minutes during the specified window.

Nextgen apologizes for any inconvenience caused.

If you require any further assistance please do not hesitate to call Nextgen SMC on 1800 099 299.

Regards

Kindest Regards

James Braunegg
W:  1300 769 972  |  M:  0488 997 207 |  D:  (03) 9751 7616
E:   james.braunegg at micron21.com<mailto:james.braunegg at micron21.com>  |  ABN:  85 126 029 203

[M21.jpg]

This message is intended for the addressee named above. It may contain privileged or confidential information. If you are not the intended recipient of this message you must not use, copy, distribute or disclose it to anyone other than the addressee. If you have received this message in error please return the message to the sender by replying to it and then delete the message from your computer.


From: Damien Coxall [mailto:damien at dreamboat.id.au]
Sent: Tuesday, March 16, 2010 1:49 PM
To: James Braunegg
Subject: Re: [AusNOG] So who broke the internet now ?

Hi James,

There was nothing unusual in my monitoring overnight, either with domestic / international transit or overseas transmission.

Both Netspace and Primus live downstream of Optus and from the looks of things so do you, so I can only guess the problem lay there somewhere, even though I saw no issue with any of my data interconnects with them.

Damien,
Vocus.
On Tue, Mar 16, 2010 at 1:20 AM, James Braunegg <james.braunegg at micron21.com<mailto:james.braunegg at micron21.com>> wrote:
Hey all, did anyone else see some international and some domestic ping times increase by 10x around 12:12am this morning? Lasting for around 15 minutes ?
We saw some of our outbound transit to "parts of the internet slow down" we routed around the issues to resolve it.
Also noticed people are complaining from many ISP's on whirlpool.....
>From our monitoring BGP was flapping around like a pig... its all seems fine now but mainly interested to know what happen...
My guess some major link went down... traffic went via redundant paths which couldn't take all the traffic hence the increased ping times...
So who's cable broke ?
Netspace issues
http://forums.whirlpool.net.au/forum-replies.cfm?t=1404190
Optus issues
http://forums.whirlpool.net.au/forum-replies.cfm?t=1404247
Primus issues
http://forums.whirlpool.net.au/forum-replies.cfm?t=1404283
Kindest Regards

James Braunegg
W:  1300 769 972  |  M:  0488 997 207 |  D:  (03) 9751 7616
E:   james.braunegg at micron21.com<mailto:james.braunegg at micron21.com>  |  ABN:  85 126 029 203

This message is intended for the addressee named above. It may contain privileged or confidential information. If you are not the intended recipient of this message you must not use, copy, distribute or disclose it to anyone other than the addressee. If you have received this message in error please return the message to the sender by replying to it and then delete the message from your computer.


_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20100316/ae490359/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2683 bytes
Desc: image001.jpg
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20100316/ae490359/attachment.jpg>