<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"><meta name="Generator" content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Arial","sans-serif";
color:navy;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:595.3pt 841.9pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.WordSection1
{page:WordSection1;}
--></style></head><body lang="EN-AU" link="blue" vlink="blue"><div class="WordSection1"><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Matt,</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks for the Info, would have been good to have this in my ticket though… I am still waiting for a reply on a few questions, in summary they are below.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Why did it take 3 hours to shut down the offending port? I called the NOC 1 hour after it started, got put through to a messaging service and I only received a call back 2 hours later, I also submitted a ticket to which I received a reply 2 hours later also.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I was not too concerned that it had happened, as these things do happen, but the inability to contact someone at PIPE to notify them of the issue is a larger concern to me.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Last time this happened it started at 2am again, but I shut our port down and waited till morning to see if PIPE resolved it but they hadn’t, so I called up 5 hours later and someone answered and had NO idea it was happening, he placed me on hold, had a look and then said to me that they can see an issue. Within minutes the issue was fixed. I’m not trying to cause a stir here, but the process / escalation of your monitoring really needs to be looked at. As many have said this is the 5<sup>th</sup> or 6<sup>th</sup> time it has happened, and EVERYTIME I have called your NOC and they have not been aware of the issue. As stated above, I am still yet to receive a response from my open ticket as to the issue.</span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1E487C">Kindest Regards,<br>Jared Hirst<br><b><br>Servers Australia Pty Ltd<br></b><br>Direct: (02) 4307 4205<br>
Fax: (02) 4307 4201<br>Network Ops: (02) 9037 4343<br>Web: </span><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#006FBF"> </span><u><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:blue"><a href="http://www.serversaustralia.com.au/">http://www.serversaustralia.com.au</a><br>
</span></u><span lang="EN-US" style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#1E487C">Office Address: 2/2 Teamster Close, Tuggerah NSW 2259<br>Postal Address: PO Box 3187, Tuggerah NSW 2259</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:13.5pt;font-family:"Helvetica","sans-serif";color:black"> </span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span></p><div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal">
<b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:ausnog-bounces@lists.ausnog.net">ausnog-bounces@lists.ausnog.net</a> [mailto:<a href="mailto:ausnog-bounces@lists.ausnog.net">ausnog-bounces@lists.ausnog.net</a>] <b>On Behalf Of </b>Matt Whitlock<br>
<b>Sent:</b> Monday, 14 November 2011 5:39 PM<br><b>To:</b> Andrew Fort; Sean K. Finn<br><b>Cc:</b> <a href="mailto:ausnog@ausnog.net">ausnog@ausnog.net</a><br><b>Subject:</b> Re: [AusNOG] SPAM-LOW: Pipe peering issue - Equinix</span></p>
</div></div><p class="MsoNormal"> </p><p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;font-family:"Courier New"">Hi everyone,</span></p><p class="MsoNormal" style="text-autospace:none">
<span style="font-size:10.0pt;font-family:"Courier New""> </span></p><p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;font-family:"Courier New"">I just wanted to make a clear statement about where we are up to on this issue.</span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;font-family:"Courier New""> </span></p><p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;font-family:"Courier New"">In the past few months we have seen problems with unknown unicast flooding on the PIPE Sydney IX. This issue affects a subset of peers connected to just one of the switches that make up the Sydney IX.</span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;font-family:"Courier New""> </span></p><p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;font-family:"Courier New"">We use a combination of port security and layer 2 ACL's on all peering interfaces to prevent this type of problem. Over the years these mechanisms have prevented many broadcast storms and loops on the network. For those concerned, our monitoring system does detect this and other similar problems for reporting to the NOC. They are monitored and escalated as required.</span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;font-family:"Courier New""> </span></p><p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;font-family:"Courier New"">With these particular issues, the controls do not appear to be effective. We suspect a hardware or software problem to be the cause, but it has proven difficult to reproduce. The issue has been narrowed to traffic injected by a specific peer and some additional protections are now in place to minimise the risk while we work with the peer to find the best permanent solution.</span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:10.0pt;font-family:"Courier New""> </span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">We have had a migration plan in progress for a while now to replace the switch platform; I’m sure you can all imagine it is not a quick or easy task to complete. We have an expectation of contacting you all in the next few days to continue this work with a view to setting a mutually agreed migration date.</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> </span></p><p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">Ta,<br><br>Matt Whitlock <br>Operations Manager, PIPE </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:navy"> </span></p><div><div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-US"><hr size="2" width="100%" align="center">
</span></div><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> <a href="mailto:ausnog-bounces@lists.ausnog.net">ausnog-bounces@lists.ausnog.net</a> <a href="mailto:[mailto:ausnog-bounces@lists.ausnog.net]">[mailto:ausnog-bounces@lists.ausnog.net]</a> <b>On Behalf Of </b>Andrew Fort<br>
<b>Sent:</b> Monday, 14 November 2011 11:43 AM<br><b>To:</b> Sean K. Finn<br><b>Cc:</b> <a href="mailto:ausnog@ausnog.net">ausnog@ausnog.net</a><br><b>Subject:</b> Re: [AusNOG] SPAM-LOW: Pipe peering issue - Equinix</span><span lang="EN-US"></span></p>
</div><p class="MsoNormal"> </p><p>IXes for years have done mac port filtering to mitigate this issue.<br>One port, one mac. No tagging. If a port is lit up, it should point to the issue (if filtered frames hit l2 int counters). A faulty transceiver spewing a valid source mac?</p>
<p>Is mac filtering not being done there, or is it not working?</p><div><p class="MsoNormal">On Nov 13, 2011 5:33 PM, "Sean K. Finn" <<a href="mailto:sean.finn@ozservers.com.au">sean.finn@ozservers.com.au</a>> wrote:</p>
<div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">When a CAM Table is maxed-out a switch becomes a full-duplex hub. (Cool hey).</span><span lang="EN-US"></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D">If it doesn’t know which mac address to send the traffic to, it sends it EVERYWHERE.</span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">If the traffic is deliberately being injected into a port knowing that the mac address doesn’t exist on the peering fabric, EVERYONE gets it, even if the CAM table isn’t full.</span><span lang="EN-US"></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D">I’m not on PIPE-IX-SYD, (Only on Equinix-SYD, O Hai) but I can imagine that..</span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">There’s a customer connected to the IX who’s own network has become a hub so they are smashing L2 traffic into the IX not on purpose.</span></b><span lang="EN-US"></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D">At least I’d hope that’s the case, I wouldn’t imagine anyone would deliberately sabotage the IX.</span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">If someone can sniff the packets you may be able to get the MAC address, then do an Arping on all of the IX IP’s and see which mac addresses match the IP’s in question, or look at your routers ARP table for a corresponding MAC address to get the offending networks PEER-IP. (Or look on the graphs and see who is inverse to everyone else).</span><span lang="EN-US"></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D">Who has more peers, by the way? Equinix-SYD or PIPE-SYD? </span><span lang="EN-US"></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:11.0pt;color:#1F497D">S.</span><span lang="EN-US"></span></p><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:11.0pt;color:#1F497D"> </span><span lang="EN-US"></span></p>
<div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:10.0pt">From:</span></b><span lang="EN-US" style="font-size:10.0pt"> <a href="mailto:ausnog-bounces@lists.ausnog.net" target="_blank">ausnog-bounces@lists.ausnog.net</a> [mailto:<a href="mailto:ausnog-bounces@lists.ausnog.net" target="_blank">ausnog-bounces@lists.ausnog.net</a>] <b>On Behalf Of </b>ZoneNetworks - Joel<br>
<b>Sent:</b> Sunday, November 13, 2011 9:35 AM<br><b>To:</b> Skeeve Stevens<br><b>Cc:</b> <a href="mailto:ausnog@ausnog.net" target="_blank">ausnog@ausnog.net</a><br><b>Subject:</b> Re: [AusNOG] SPAM-LOW: Pipe peering issue - Equinix</span><span lang="EN-US"></span></p>
</div></div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">To my knowledge it's a hardware/firmware issue not traffic related ...though looking at the graphs it seems like traffic/flood which is misleading</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><span lang="EN-US">If more customers gave them a hard time it might get fixed sooner, we were told (in august) they had a long term fix in plan but no ETA when it will be done</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Not something you would hear from the "old pipe"</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Joel</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">Sent from my iPad</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><span lang="EN-US"><br>On 13/11/2011, at 10:06 AM, Skeeve Stevens <<a href="mailto:Skeeve@eintellego.net" target="_blank">Skeeve@eintellego.net</a>> wrote:</span></p>
</div><blockquote style="margin-top:5.0pt;margin-bottom:5.0pt"><div><div><div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Do we know where the traffic is coming from?</span></p>
</div><div><div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:13.5pt;color:black"> </span><span lang="EN-US"></span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:13.5pt;color:#002060">…Skeeve</span><span lang="EN-US"></span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:13.5pt;color:black"> </span><span lang="EN-US"></span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:13.5pt;color:#002060">--</span><span lang="EN-US"></span></p></div><div><div><p style="margin:0cm;margin-bottom:.0001pt">
<span lang="EN-US" style="font-size:10.0pt;color:black">Skeeve Stevens, CEO - eintellego Pty Ltd</span><span lang="EN-US"></span></p><p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US" style="font-size:10.0pt;color:black"><a href="mailto:skeeve@eintellego.net" target="_blank">skeeve@eintellego.net</a> ; <a href="http://www.eintellego.net" target="_blank">www.eintellego.net</a></span><span lang="EN-US"></span></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US" style="font-size:10.0pt;color:black">Phone: 1300 753 383 ; Fax: <a href="tel:%28%2B612%29%208572%209954" target="_blank">(+612) 8572 9954</a></span><span lang="EN-US"></span></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US" style="font-size:10.0pt;color:black">Cell <a href="tel:%2B61%20%280%29414%20753%20383" target="_blank">+61 (0)414 753 383</a> ; skype://skeeve</span><span lang="EN-US"></span></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US" style="font-size:10.0pt;color:black"><a href="http://facebook.com/eintellego" target="_blank">facebook.com/eintellego</a></span><span lang="EN-US"></span></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US" style="font-size:10.0pt;color:black"><a href="http://twitter.com/networkceoau" target="_blank">twitter.com/networkceoau</a> ; <a href="http://www.linkedin.com/in/skeeve" target="_blank">www.linkedin.com/in/skeeve</a></span><span lang="EN-US"></span></p>
<p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US" style="font-size:10.0pt;color:black">PO Box 7726, Baulkham Hills, NSW 1755 Australia</span><span lang="EN-US"></span></p><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US" style="font-size:13.5pt;color:#002060"> </span><span lang="EN-US"></span></p></div><p style="margin:0cm;margin-bottom:.0001pt"><span lang="EN-US" style="font-size:10.0pt;color:black">--</span><span lang="EN-US"></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:13.5pt;color:#7F007F">eintellego - The Experts Who The Experts Call</span><span lang="EN-US"></span></p></div>
<div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US" style="font-size:13.5pt;color:#002060">Juniper - HP Networking - Cisco - Brocade</span><span lang="EN-US"></span></p>
</div></div></div></div></div></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">On 13/11/11 10:04 AM, "Jared Hirst" <<a href="mailto:jared.hirst@serversaustralia.com.au" target="_blank">jared.hirst@serversaustralia.com.au</a>> wrote:</span></p></div></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div><blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt"><div><div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">Yeah every few weeks it happens is what I mean, they start sending</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">hundreds of mbits of useless broadcast traffic to all customers in</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Equinix. It's getting frustrating and annoying as it always seems to</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">happen at 2:00on a Saturday morning :(</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">Kindest Regards,</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Jared Hirst</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Servers Australia Pty Ltd</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Phone: 02 4307 4200</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">Fax: 02 4307 4201</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Web: <a href="http://www.serversaustralia.com.au" target="_blank">http://www.serversaustralia.com.au</a></span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">On 13/11/2011, at 4:26 AM, "<a href="mailto:joel@zonenetworks.com.au" target="_blank">joel@zonenetworks.com.au</a>"</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"><<a href="mailto:joel@zonenetworks.com.au" target="_blank">joel@zonenetworks.com.au</a>> wrote:</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><blockquote style="border:none;border-left:solid #B5C4DF 4.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">It's been happening for months now not few weeks</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Last time we were told they need to fix switch/router but they can't do it for some reason</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Sent from my HTC</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">----- Reply message -----</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">From: "Jared Hirst" <<a href="mailto:jared.hirst@serversaustralia.com.au" target="_blank">jared.hirst@serversaustralia.com.au</a>></span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">To: "<a href="mailto:ausnog@ausnog.net" target="_blank">ausnog@ausnog.net</a>" <<a href="mailto:ausnog@ausnog.net" target="_blank">ausnog@ausnog.net</a>></span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Subject: SPAM-LOW: [AusNOG] Pipe peering issue - Equinix</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">Date: Sun, Nov 13, 2011 3:30 am</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Guys,</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Just as an FYI there appears to be some form of broadcast storm on the</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">pipe peering fabric in Equinix Sydney, looks to be affecting a few</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">people and filling links from the looks of their online graphs.</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div>
<div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">We have had to shut our pipe peering down until they resolve it, which</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">is my next question... Do pipe have a 24/7 NOC still? I called over 30</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">minutes ago and went to a messaging service but have not had a call</span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">back, have also created a ticket with no response.</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">So if anyone is wondering why their pipe link is full, this is why.</span></p></div>
<div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Appears to be happening every few weeks in Equinix lately.</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Kindest Regards,</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">Jared Hirst</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">Servers Australia Pty Ltd</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">Phone: 02 4307 4200</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">Fax: 02 4307 4201</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">Web: <a href="http://www.serversaustralia.com.au" target="_blank">http://www.serversaustralia.com.au</a></span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US">_______________________________________________</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">AusNOG mailing list</span></p></div>
<div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"><a href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a></span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"><a href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a></span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">
<span lang="EN-US"> </span></p></div></blockquote><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">_______________________________________________</span></p></div><div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US">AusNOG mailing list</span></p></div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"><a href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a></span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"><a href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a></span></p>
</div><div><p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span lang="EN-US"> </span></p></div></div></div></blockquote></div></blockquote></div></div><p class="MsoNormal" style="margin-bottom:12.0pt">
<br>_______________________________________________<br>AusNOG mailing list<br><a href="mailto:AusNOG@lists.ausnog.net">AusNOG@lists.ausnog.net</a><br><a href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a></p>
</div></div></body></html>