[AusNOG] Broadcast Storm - IX NextDC S1

[Matt Perkins]

The more relevant question is. Why was mitigation implemented after the last time this happened. 



-- 
/* Matt Perkins
       Direct 1300 137 379     Spectrum Networks Ptd. Ltd.
       Office 1300 133 299     matt at spectrum.com.au
       Fax    1300 133 255     Level 6, 350 George Street Sydney 2000
      SIP 1300137379 at sip.spectrum.com.au
       Google Talk MattAPerkins at gmail.com
       PGP/GNUPG Public Key can be found at  http://pgp.mit.edu
*/

> On 23 Sep 2020, at 4:58 pm, Abhi Gudidevuni <abhi1g at gmail.com> wrote:
> 
> 
> Hi Ryan,
> 
> The Sydney Mega IX experienced a customer initiated broadcast storm. We are investigating mitigation actions. If you have not already, please lodge a support request and the team will provide a report.
> 
> Regards,
> 
> --
> Abhi Gudidevuni
> Head of Customer & Network Services
> 
> 
>> On Wed, Sep 23, 2020 at 4:37 PM Karl Hardisty <karl at mothership.co.nz> wrote:
>> Hi Ryan,
>> 
>> You’re a little late to the party; there was an ARP storm on Mega-IX Sydney yesterday. 
>> 
>> Much hilarity ensued. Particularly if peers had default Juniper ARP policers. 
>> 
>> TL:DR it wasn’t just you. It was most of Au and NZ (sucking at the teat of MegaIX in Sydney). 
>> 
>> Kind regards,
>> 
>> Karl
>> 
>> lE karl at mothership.co.nz lW mothership.co.nz  lA PO Box 99814, Newmarket  lM 021 999 990 lP 09 974 3171
>> 
>> 
>> 
>> 
>> 
>>> On 23/09/2020, at 6:32 PM, Ryan van Klaveren <ryan.vanklaveren at comvergence.com.au> wrote:
>>> 
>>> Hey all,
>>> 
>>> Did anyone else experience any irregularities yesterday (22 Sept) between 10:39am - 11:50am ? (I know that sounds a bit vague) - but this is what we experienced during this window:
>>> 
>>> * Our nameservers failing to do resolution
>>> * Sydney transit routers had BGP flaps to IX NSW peers
>>> * Sydney ASR router generated 143 logs of: PUNT_INJECT-5-DROP_PUNT_CAUSE: punt cause policer drop packet casue 7 (ARP request or response)
>>> 
>>> Spoke to IX, who said: 
>>> "Yep we did, yesterday another IX had an outage (I believe a broadcast storm occurred) around the timestamps in your logs, a number of our members that had remote ends also connected to the other IX were affected by that outage and that is likely the reason for their session to flap towards us" 
>>> 
>>> 
>>> Just trying to see if anyone else experienced it, and better yet have any more details as to what may have caused it?
>>> 
>>> 
>>> 
>>> Kind regards,
>>> Ryan van Klaveren
>>> 
>>> Main: 1300 550 125
>>> Fax: 1300 550 121
>>> 
>>> 
>>>        www.comvergence.com.au 
>>> Level 2, 38-40 Garden Street, South Yarra, VIC 3141
>>> 
>>> 
>>> Converging Tomorrow Today
>>> Confidentiality Note: This e-mail is sent to and intended for use by the named addressees only. It contains confidential information. If you receive this e-mail in error, please telephone Comvergence on 1300 550 125, and then delete this message immediately. Further, you should not disclose, print, re-transmit, copy, store, or reveal the contents of this message to any third party.
>>> 
>>> 
>>> _______________________________________________
>>> AusNOG mailing list
>>> AusNOG at lists.ausnog.net
>>> http://lists.ausnog.net/mailman/listinfo/ausnog
>> 
>> _______________________________________________
>> AusNOG mailing list
>> AusNOG at lists.ausnog.net
>> http://lists.ausnog.net/mailman/listinfo/ausnog
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20200923/ffacbfcf/attachment.html>