<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1261379593;
mso-list-type:hybrid;
mso-list-template-ids:-148046980 -1822795226 201916419 201916421 201916417 201916419 201916421 201916417 201916419 201916421;}
@list l0:level1
{mso-level-start-at:4;
mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style>
</head>
<body lang="EN-AU" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">Hi Everyone,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">I’ve been looking at improving our BGP configuration lately, and I would just like to see if I’m missing anything obvious in terms of speeding up BGP convergence (particularly inbound convergence)
with our transit providers during failover. I understand that BGP convergence on the internet is not going to be perfect, but I am trying to ensure I tune things as best I can. We are using Cisco ASR1001-Xs for reference, though I’m more wondering about general
best practices that other ISPs use, that I can then adapt to our network.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">I’m also curious if my expectations of trying to minimise convergence times with transit peers are realistic or not.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">Right now, I am seeing 20-30 second outage windows when failing over my announced prefixes from one transit provider to another. I can understand this when transitioning between transits, but
I see this even when failing over between a primary/secondary peering session with a single AS / transit provider, which is disappointing. My hope was to have almost no interruption where we have multiple links with a given transit provider, and small convergence
window (maybe 2-5s?) when transitioning prefixes from one transit to another.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">With iBGP seems like there’s lots of options and it would be possible to achieve sub-second convergence fairly easily. But eBGP is where it becomes more limited and difficult to improve the situation.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-GB" style="font-size:11.0pt">For iBGP I can do:<o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></b></p>
<ul style="margin-top:0cm" type="disc">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="font-size:11.0pt">BFD
<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="font-size:11.0pt">BGP Multipath – I haven’t tested, but I assume having multiple paths in the FIB will speed up failover convergence.<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="font-size:11.0pt">BGP Best External<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="font-size:11.0pt">Add Path<o:p></o:p></span></li></ul>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-GB" style="font-size:11.0pt">For eBGP I can do:<o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></b></p>
<ul style="margin-top:0cm" type="disc">
<li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="font-size:11.0pt">BFD (If supported by the upstream – I have this on all peers)<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="font-size:11.0pt">Advertisement Internal – I have set this to 0 (doesn’t make a major difference, but helps a little)<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="font-size:11.0pt">BGP Multipath (if supported by the upstream – unfortunately my upstream requires the primary/secondary paths are enforced on their side
via localpref so I can’t leverage this).<o:p></o:p></span></li><li class="MsoListParagraph" style="margin-left:0cm;mso-list:l0 level1 lfo1"><span lang="EN-GB" style="font-size:11.0pt">AS Path prepending of the same prefixes instead of announcing less/more specific prefixes at different sites seems to help.<o:p></o:p></span></li></ul>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">I haven’t found any other commonly accepted methods of announcing a backup path to eBGP peers.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">We are using Equinix Connect transit in Sydney as our main transit, where we have primary and secondary links between us and Equinix. And Vocus as our main transit in Melbourne, with the intention
of failing over all our announced prefixes between sites as required, by leveraging AS Path prepending.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt">Are there any other techniques or best practices I am missing to help try and reduce downtime in the event of a router or BGP session failure event?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;color:black">Appreciate any insights you can offer, and hope this proves to be a useful and interesting discussion for others.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;color:black">Thanks!<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;color:black"><br>
Rhys Hanrahan<br>
Chief Information Officer<br>
Nexus One Pty Ltd<br>
<br>
E: <a href="mailto:support@nexusone.com.au" target="_blank"><span style="color:#0563C1">support@nexusone.com.au</span></a><br>
P: +61 2 9191 0606<br>
W: http://www.nexusone.com.au/<br>
M: PO Box 127, Royal Exchange NSW 1225<br>
A: Level 10 307 Pitt St, Sydney NSW 2000<br>
<br>
</span><span lang="EN-GB" style="font-size:11.0pt;color:black;mso-fareast-language:EN-AU"><img border="0" width="280" height="73" style="width:2.9166in;height:.7604in" id="Picture_x0020_2" src="cid:image001.jpg@01D3AEFC.A6960EC0" alt="ttp://quintus.nexusone.com.au/~rhys/nexus1-email-sig.jpg"></span><span lang="EN-GB"><o:p></o:p></span></p>
</div>
</body>
</html>