[AusNOG] International link issue

Fri Feb 24 10:30:14 EST 2012

Telstra claims they had an international link down:

http://www.zdnet.com.au/telstra-hit-by-nationwide-data-outage-339332310.htm

If that happened at the same time as DODO incorrectly sending Telstra the
full BGP table, could that explain why Telstra black-holed all-routes plus
pumped all of it's own traffic via dodo?

On 24 February 2012 10:02, Wade Millican <Wade.Millican at echoent.com.au>wrote:

> Hi All,
>
> What I'm yet to understand about this outage is why DODO's AS_PATH was
> seen as shorter than anything Telstra already had.
>
> An earlier posted look at routes(below), thanks Gavin, shows all routes
> from Telstra taking hops to DODO, then Optus or PIPE before moving to the
> destination. Surely Telstra would have had better routes than pushing all
> traffic 2 hops out of it's way.
>
> AS_PATH does not explain how Telstra accepted these as the active routes.
> Even if all routes were accepted, Telstra still has better routes.
>
> Can anyone explain what BGP Metric was modified/used that pushed traffic
> over longer AS_PATHs?
>
> *> 1.22.161.0/24    165.228.157.73         100     80      0 1221 38285 7474 7473 55410 45528 i
> *> 1.22.162.0/24    165.228.157.73         100     80      0 1221 38285 7474 7473 55410 45528 i
> *> 1.22.163.0/24    165.228.157.73         100     80      0 1221 38285 7474 7473 55410 45528 i
> *> 1.22.167.0/24    165.228.157.73         100     80      0 1221 38285 7474 7473 6453 4755 45528 i
> *> 1.22.168.0/24    165.228.157.73         100     80      0 1221 38285 7474 7473 6453 4755 45528 i
> ..
> *  14.201.64.0/24   165.228.157.73         100     80      0 1221 38285 18398 7545 7545 i
>
>
> Thanks,
>
> Wade
> --
> Wade Millican
> Technical Consultant Team Lead
> Hemisphere Infrastructure Support
> Information Technology
> *Echo Entertainment Group Limited*
>
> 2 Edward St
> Pyrmont NSW 2009
>
> T: +61 2 9657 7460
> M: +61 (0) 400 192 485
> *wade.millican at echoent.com.au
> *www.echoentertainment.com.au
> From: "Ramsay, Paul" <pramsay at uecomm.com.au>
> Date: Wed, 22 Feb 2012 22:20:41 -0800
> To: "ausnog at ausnog.net" <ausnog at ausnog.net>
> Subject: Re: [AusNOG] International link issue
>
> Yes, this reinforces the Rule of Trust. Don’t trust your BGP peers and
> ensure your filters are in place, configured correctly and working, you
> can’t transfer blame.****
>
> It can cost you big $$ and pain if you inadvertently turn yourself into a
> transit peer because your upstreams may prefer to send traffic where they
> can make $$ from.****
>
> ** **
>
> *From:* ausnog-bounces at lists.ausnog.net [
> mailto:ausnog-bounces at lists.ausnog.net <ausnog-bounces at lists.ausnog.net>]
> *On Behalf Of *Sean K. Finn
> *Sent:* Thursday, 23 February 2012 5:09 PM
> *To:* 'ausnog at ausnog.net'
> *Subject:* Re: [AusNOG] International link issue****
>
> ** **
>
> It’s easy to describe for all the media types watching..****
>
> (And I’m not sure why its not being put out there in Laymans terms).****
>
> ** **
>
> From the routes seen at various points, and reported on the WAIX mailing
> list earlier..****
>
> ** **
>
> ** **
>
> ** **
>
> Dodo told Telstra that Dodo was the rest of the Internet.****
>
> ** **
>
> Telstra Believed Dodo.****
>
> ** **
>
> Telstra entire system tried to use DODO as their ISP instead of everyone
> else Telstra is connected to.****
>
> ** **
>
> Needless to say this didn’t work, the pipes got Jammed.****
>
> ** **
>
> Telstra should have filtered the announcement from Dodo, butdidn’t.****
>
> ** **
>
> Filtering is in place as a form of control (which is used instead of
> trust).****
>
> ** **
>
> Filtering obviously wasn’t in place, or didn’t work, so anything that Dodo
> told Telstra about where to find the Internet, Telstra believed.****
>
> ** **
>
> This happens quite often, I’ve heard of this happening on peering
> exchanges within Australia, too. Just never at an organizational level as
> big as Telstra.****
>
> ** **
>
> Over and Out.****
>
> ** **
>
> ** **
>
> This message and its attachments may contain legally privileged or
> confidential information. It is for the intended addressee(s) only.****
>
> If you are not the intended recipient you must not disclose or use the
> information contained in it. If you have received this email in error
> please notify us immediately by return email and delete the document.****
>
> Any views expressed in this message are those of the individual sender,
> except where the sender specifies and with authority, states them to be the
> views of the Company.****
>
> Uecomm accepts no liability for any damage caused by this email or its
> attachments due to viruses, interference, interception, corruption or
> unauthorised access.****
>
> ------------------------------
> This e-mail message has been scanned for Viruses and Content and cleared
> by *NetIQ MailMarshal *
> ------------------------------
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20120224/5fc88a3e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 78BFFC55-58BE-42A5-94D5-509927E7B33A.png
Type: image/png
Size: 13740 bytes
Desc: not available
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20120224/5fc88a3e/attachment.png>