[AusNOG] So who broke the internet now ?

[Colwell, Scott]

No.  Changing a line card on an SDH add drop mux doesn't normally even impact a single
ISP.

Scott

________________________________
From: ausnog-bounces at lists.ausnog.net [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Darren Moss
Sent: Tuesday, 16 March 2010 3:27 PM
To: James Braunegg; Damien Coxall
Cc: ausnog at ausnog.net
Subject: Re: [AusNOG] So who broke the internet now ?

Wow, so 1 maintenance activity by a single provider changing a network interface caused all that uproar and poor international performance between multiple service providers and their customers.

Can that be right ?


Regards,


Darren Moss
General Manager
Australia and New Zealand
[p] 1300 131 083 [f] 03 9017 2287
[e] Darren.Moss at em3.com.au<mailto:Darren.Moss at em3.com.au> [w] www.em3.com.au<http://www.em3.com.au/>

em3 People and Technology | Managed Technology Experts
postal: PO Box 2333, Moorabbin VIC 3189

New Zealand Airedale Street, Auckland City
postal: PO Box 39573, Howick 2045
[p] 09 887 0550 [f] 09 887 0273



________________________________
From: ausnog-bounces at lists.ausnog.net [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of James Braunegg
Sent: 2010-03-16 3:20 pm
To: Damien Coxall
Cc: ausnog at ausnog.net
Subject: Re: [AusNOG] So who broke the internet now ?

Found the source of the issue last night... Next Gen were doing some upgrades... which went a little wrong...

Dear Customer,

Please be advised the following Emergency Planned Outage Works by Nextgen Network has been scheduled.

This work is expected to result in an outage to your service/s listed below.

Start Date:       16/03/10            Start Time:      00:01 (AEDT)

Finish Date:     16/03/10            Finish Time:    02:00 (AEDT)

Description of Works:

Nextgen has organized an emergency planned outage to replace the faulty card on one of the core device.

Your service listed above will experience an outage of 30 minutes during the specified window.

Nextgen apologizes for any inconvenience caused.

If you require any further assistance please do not hesitate to call Nextgen SMC on 1800 099 299.

Regards

Kindest Regards

James Braunegg
W:  1300 769 972  |  M:  0488 997 207 |  D:  (03) 9751 7616
E:   james.braunegg at micron21.com<mailto:james.braunegg at micron21.com>  |  ABN:  85 126 029 203

[cid:836221223 at 17032010-18D0]

This message is intended for the addressee named above. It may contain privileged or confidential information. If you are not the intended recipient of this message you must not use, copy, distribute or disclose it to anyone other than the addressee. If you have received this message in error please return the message to the sender by replying to it and then delete the message from your computer.


From: Damien Coxall [mailto:damien at dreamboat.id.au]
Sent: Tuesday, March 16, 2010 1:49 PM
To: James Braunegg
Subject: Re: [AusNOG] So who broke the internet now ?

Hi James,

There was nothing unusual in my monitoring overnight, either with domestic / international transit or overseas transmission.

Both Netspace and Primus live downstream of Optus and from the looks of things so do you, so I can only guess the problem lay there somewhere, even though I saw no issue with any of my data interconnects with them.

Damien,
Vocus.
On Tue, Mar 16, 2010 at 1:20 AM, James Braunegg <james.braunegg at micron21.com<mailto:james.braunegg at micron21.com>> wrote:
Hey all, did anyone else see some international and some domestic ping times increase by 10x around 12:12am this morning? Lasting for around 15 minutes ?
We saw some of our outbound transit to "parts of the internet slow down" we routed around the issues to resolve it.
Also noticed people are complaining from many ISP's on whirlpool.....
>From our monitoring BGP was flapping around like a pig... its all seems fine now but mainly interested to know what happen...
My guess some major link went down... traffic went via redundant paths which couldn't take all the traffic hence the increased ping times...
So who's cable broke ?
Netspace issues
http://forums.whirlpool.net.au/forum-replies.cfm?t=1404190
Optus issues
http://forums.whirlpool.net.au/forum-replies.cfm?t=1404247
Primus issues
http://forums.whirlpool.net.au/forum-replies.cfm?t=1404283
Kindest Regards

James Braunegg
W:  1300 769 972  |  M:  0488 997 207 |  D:  (03) 9751 7616
E:   james.braunegg at micron21.com<mailto:james.braunegg at micron21.com>  |  ABN:  85 126 029 203

This message is intended for the addressee named above. It may contain privileged or confidential information. If you are not the intended recipient of this message you must not use, copy, distribute or disclose it to anyone other than the addressee. If you have received this message in error please return the message to the sender by replying to it and then delete the message from your computer.


_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************


_____________________________________________________________________ 
This e-mail has been scanned for viruses by MCI's Internet Managed 
Scanning Services - powered by MessageLabs. For further information 
visit http://www.mci.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20100318/5a583819/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 2683 bytes
Desc: image001.jpg
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20100318/5a583819/attachment.jpg>