<div dir="ltr"><div><div>Hello Rhys,<br><br>David Hughes presented a paper on BGP tuning some years ago (at AusNOG?) which may be worth digging up. While there have been few changes to BGP itself, of course there have been bells and whistles added to the routers (such as BFD, etc.) which will help so I guess the paper may be (over) due for an update (David? :-).<br><br></div>Cheers, Chris Chaundy<br></div>(Retired Network Engineer)<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Feb 26, 2018 at 12:23 PM, Rhys Hanrahan <span dir="ltr"><<a href="mailto:rhys@nexusone.com.au" target="_blank">rhys@nexusone.com.au</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="#0563C1" vlink="#954F72" lang="EN-AU">
<div class="m_-2477653671737839675WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB">Hi Everyone,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB">I’ve been looking at improving our BGP configuration lately, and I would just like to see if I’m missing anything obvious in terms of speeding up BGP convergence (particularly inbound convergence)
with our transit providers during failover. I understand that BGP convergence on the internet is not going to be perfect, but I am trying to ensure I tune things as best I can. We are using Cisco ASR1001-Xs for reference, though I’m more wondering about general
best practices that other ISPs use, that I can then adapt to our network.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB">I’m also curious if my expectations of trying to minimise convergence times with transit peers are realistic or not.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB">Right now, I am seeing 20-30 second outage windows when failing over my announced prefixes from one transit provider to another. I can understand this when transitioning between transits, but
I see this even when failing over between a primary/secondary peering session with a single AS / transit provider, which is disappointing. My hope was to have almost no interruption where we have multiple links with a given transit provider, and small convergence
window (maybe 2-5s?) when transitioning prefixes from one transit to another.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB">With iBGP seems like there’s lots of options and it would be possible to achieve sub-second convergence fairly easily. But eBGP is where it becomes more limited and difficult to improve the situation.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt" lang="EN-GB">For iBGP I can do:<u></u><u></u></span></b></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></b></p>
<ul style="margin-top:0cm" type="disc">
<li class="m_-2477653671737839675MsoListParagraph" style="margin-left:0cm"><span style="font-size:11.0pt" lang="EN-GB">BFD
<u></u><u></u></span></li><li class="m_-2477653671737839675MsoListParagraph" style="margin-left:0cm"><span style="font-size:11.0pt" lang="EN-GB">BGP Multipath – I haven’t tested, but I assume having multiple paths in the FIB will speed up failover convergence.<u></u><u></u></span></li><li class="m_-2477653671737839675MsoListParagraph" style="margin-left:0cm"><span style="font-size:11.0pt" lang="EN-GB">BGP Best External<u></u><u></u></span></li><li class="m_-2477653671737839675MsoListParagraph" style="margin-left:0cm"><span style="font-size:11.0pt" lang="EN-GB">Add Path<u></u><u></u></span></li></ul>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt" lang="EN-GB">For eBGP I can do:<u></u><u></u></span></b></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></b></p>
<ul style="margin-top:0cm" type="disc">
<li class="m_-2477653671737839675MsoListParagraph" style="margin-left:0cm"><span style="font-size:11.0pt" lang="EN-GB">BFD (If supported by the upstream – I have this on all peers)<u></u><u></u></span></li><li class="m_-2477653671737839675MsoListParagraph" style="margin-left:0cm"><span style="font-size:11.0pt" lang="EN-GB">Advertisement Internal – I have set this to 0 (doesn’t make a major difference, but helps a little)<u></u><u></u></span></li><li class="m_-2477653671737839675MsoListParagraph" style="margin-left:0cm"><span style="font-size:11.0pt" lang="EN-GB">BGP Multipath (if supported by the upstream – unfortunately my upstream requires the primary/secondary paths are enforced on their side
via localpref so I can’t leverage this).<u></u><u></u></span></li><li class="m_-2477653671737839675MsoListParagraph" style="margin-left:0cm"><span style="font-size:11.0pt" lang="EN-GB">AS Path prepending of the same prefixes instead of announcing less/more specific prefixes at different sites seems to help.<u></u><u></u></span></li></ul>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB">I haven’t found any other commonly accepted methods of announcing a backup path to eBGP peers.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB">We are using Equinix Connect transit in Sydney as our main transit, where we have primary and secondary links between us and Equinix. And Vocus as our main transit in Melbourne, with the intention
of failing over all our announced prefixes between sites as required, by leveraging AS Path prepending.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB">Are there any other techniques or best practices I am missing to help try and reduce downtime in the event of a router or BGP session failure event?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black" lang="EN-GB">Appreciate any insights you can offer, and hope this proves to be a useful and interesting discussion for others.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black" lang="EN-GB"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black" lang="EN-GB">Thanks!<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;color:black" lang="EN-GB"><br>
Rhys Hanrahan<br>
Chief Information Officer<br>
Nexus One Pty Ltd<br>
<br>
E: <a href="mailto:support@nexusone.com.au" target="_blank"><span style="color:#0563c1">support@nexusone.com.au</span></a><br>
P: <a href="tel:+61%202%209191%200606" value="+61291910606" target="_blank">+61 2 9191 0606</a><br>
W: <a href="http://www.nexusone.com.au/" target="_blank">http://www.nexusone.com.au/</a><br>
M: PO Box 127, Royal Exchange NSW 1225<br>
A: Level 10 307 Pitt St, Sydney NSW 2000<br>
<br>
</span><span style="font-size:11.0pt;color:black" lang="EN-GB"><img style="width:2.9166in;height:.7604in" id="m_-2477653671737839675Picture_x0020_2" src="cid:image001.jpg@01D3AEFC.A6960EC0" alt="ttp://quintus.nexusone.com.au/~rhys/nexus1-email-sig.jpg" width="280" height="73" border="0"></span><span lang="EN-GB"><u></u><u></u></span></p>
</div>
</div>
<br>______________________________<wbr>_________________<br>
AusNOG mailing list<br>
<a href="mailto:AusNOG@lists.ausnog.net">AusNOG@lists.ausnog.net</a><br>
<a href="http://lists.ausnog.net/mailman/listinfo/ausnog" rel="noreferrer" target="_blank">http://lists.ausnog.net/<wbr>mailman/listinfo/ausnog</a><br>
<br></blockquote></div><br></div>