[AusNOG] Broadcast Storm - IX NextDC S1

[David Dunn]

Hi all,

I’d love others to provide their thoughts regarding what happened during this time as I was quite surprised to see this not be a thread at the actual time of occurrence.

Multiple providers were posting network notifications at the time that there were issues (Micron21, Zettagrid, Vultr) but no one has elaborated as to what the actual cause of the (what appears to be) peering issues were (BGP flapping, etc).

Are we sure this was just limited to MegaIX Sydney? Or was it a peer doing this over multiple Sydney based IX’s with a common peer? Who was the peer that caused this?

Looks like you have done your research Karl so please share 😊

Kind regards,
David

From: AusNOG <ausnog-bounces at lists.ausnog.net> On Behalf Of Karl Hardisty
Sent: Wednesday, 23 September 2020 4:38 PM
To: Ryan van Klaveren <ryan.vanklaveren at comvergence.com.au>
Cc: Ausnog <ausnog at lists.ausnog.net>
Subject: Re: [AusNOG] Broadcast Storm - IX NextDC S1

Hi Ryan,

You’re a little late to the party; there was an ARP storm on Mega-IX Sydney yesterday.

Much hilarity ensued. Particularly if peers had default Juniper ARP policers.

TL:DR it wasn’t just you. It was most of Au and NZ (sucking at the teat of MegaIX in Sydney).

Kind regards,

Karl

lE karl at mothership.co.nz<mailto:karl at mothership.co.nz> lW mothership.co.nz<http://mothership.co.nz/>  lA PO Box 99814, Newmarket  lM 021 999 990 lP 09 974 3171





On 23/09/2020, at 6:32 PM, Ryan van Klaveren <ryan.vanklaveren at comvergence.com.au<mailto:ryan.vanklaveren at comvergence.com.au>> wrote:

Hey all,

Did anyone else experience any irregularities yesterday (22 Sept) between 10:39am - 11:50am ? (I know that sounds a bit vague) - but this is what we experienced during this window:

* Our nameservers failing to do resolution
* Sydney transit routers had BGP flaps to IX NSW peers
* Sydney ASR router generated 143 logs of: PUNT_INJECT-5-DROP_PUNT_CAUSE: punt cause policer drop packet casue 7 (ARP request or response)

Spoke to IX, who said:
"Yep we did, yesterday another IX had an outage (I believe a broadcast storm occurred) around the timestamps in your logs, a number of our members that had remote ends also connected to the other IX were affected by that outage and that is likely the reason for their session to flap towards us"


Just trying to see if anyone else experienced it, and better yet have any more details as to what may have caused it?



Kind regards,
Ryan van Klaveren

Main: 1300 550 125
Fax: 1300 550 121
[http://www.comvergence.com.au/wp-content/themes/comvergence/img/logo.gif]
       www.comvergence.com.au<http://www.comvergence.com.au/>
Level 2, 38-40 Garden Street, South Yarra, VIC 3141

Converging Tomorrow Today
Confidentiality Note: This e-mail is sent to and intended for use by the named addressees only. It contains confidential information. If you receive this e-mail in error, please telephone Comvergence on 1300 550 125, and then delete this message immediately. Further, you should not disclose, print, re-transmit, copy, store, or reveal the contents of this message to any third party.

_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20200923/cec18575/attachment.html>