<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    <div class="moz-cite-prefix">I appreciate Jame's honesty it would
      make me want to buy from Vocus. Not the other way around. If this
      were another big wholesale provider that will remain nameless They
      would still be denying the whole thing ever happened and asking
      the customer to reset there router. <br>
      <br>
      What I am interested to know is technically what happened at the
      DC that fried stuff. I ask out of concern that if a big US data
      center can have something like this happen perhaps other's have
      the same vulnerability  (me even :)  <br>
      <br>
      I have only ever heard of things like that happening when  someone
      goes and removes a neutral from 3 phase power systems on a UPS. Or
      a change over switch doesn't overlap the neutral before switching.
        I guess if someone touched the 3 phase bus together that might
      do it but  doubtful you would break line cards just PS's  Perhaps
      a floating Telecommunications reference conductor.  It could be a
      million things.  Any more info would be great. <br>
      <br>
      Matt<br>
       <br>
      <br>
       On 27/08/2014 8:21 am, James Spenceley wrote:<br>
    </div>
    <blockquote
      cite="mid:7B7209BD-6A1E-4580-8F27-8974EB25D6F3@iroute.org"
      type="cite">
      <meta http-equiv="content-type" content="text/html;
        charset=ISO-8859-1">
      <div>Hi Wolfgang,</div>
      <div><br>
      </div>
      <div>RFP will be out to today.</div>
      <div><br>
      </div>
      <div>The long and short if it is...</div>
      <div><br>
      </div>
      <div>We had an issue caused by the DC provider which took out our
        primary pop (power surge or something similar which killed all
        the RPs and line cards). In one of those one in a million
        chances the big upgrade we'd completed across all our US sites
        left us with a code on the secondary site routers that had a bug
        which has now been diagnosed by Cisco (hence the delay in RFP),
        this bug only happens when large traffic loads (and routes) hit
        the RP from a steady state and the bug causes intermittent black
        holing of traffic passing that box.</div>
      <div><br>
      </div>
      <div>As a few people said , stuff breaks, this was one of the
        nightmare events where we had two issues in parallel one an
        undocumented Cisco bug that ironically only shows up when the
        other site failed. As I said it's a pretty rare case of events
        that had to happen to cause the outage and it took quiet a bit
        of time for Cisco to recreate/identify the bug (the other issue
        was complete hardware failure was clearly easy to diagnose),
        that Cisco delay is what has caused the delay in the RFP.</div>
      <div><br>
      </div>
      <div>James</div>
      <div><br>
      </div>
      <div><br>
        On 26 Aug 2014, at 23:58, "Wolfgang Nagele (AusRegistry)" <<a
          moz-do-not-send="true"
          href="mailto:wolfgang.nagele@ausregistry.com.au">wolfgang.nagele@ausregistry.com.au</a>>
        wrote:<br>
        <br>
      </div>
      <blockquote type="cite">
        <div>
          <meta http-equiv="Content-Type" content="text/html;
            charset=ISO-8859-1">
          <div>Hi,</div>
          <div><br>
          </div>
          <div>This is your choice. Not mine and I am sure there are
            others here that do not agree with your idea of brushing
            issues like this off the table. If this were Telstra you
            would be striking a different tone.</div>
          <div><br>
          </div>
          <div>It’s irrelevant if you have multiple upstreams - there
            are minimum requirements that we put on suppliers in the
            year 2014. Having Vocus suffer a 4 hour degradation due to a
            single DC fault in SJC is not something that we accept in
            2014. There is redundancy via LA (and Singapore) which
            didn’t work. I would like to know why - not why it didn’t
            fail over automatically there can be many reasons. Vocus
            engineers should have been able to re-route via LA and
            completely take SJC out of the equation. There are questions
            to be answered here. As well as delays in notifications - we
            have not received a notification for over an hour. Again not
            acceptable for an incident of that magnitude.</div>
          <div><br>
          </div>
          <div>We all learn based on mistakes - ignoring them gains
            nothing.</div>
          <div><br>
          </div>
          <div>As for multiple upstreams, yes we have them and yes we
            routed around the issue. To me that’s irrelevant to the
            issue at hand. If you are happy with a supplier that has 4
            hour degradation due to a single DC fault on it’s main
            international backhaul - yes - move along, nothing to see
            here.</div>
          <div><br>
          </div>
          <div>Cheers,</div>
          <div>Wolfgang</div>
          <div><br>
          </div>
          <span id="OLK_SRC_BODY_SECTION">
            <div>
              <div>On 8/26/14, 11:40 PM, "Skeeve Stevens" <<a
                  moz-do-not-send="true"
                  href="mailto:skeeve+ausnog@eintellegonetworks.com">skeeve+ausnog@eintellegonetworks.com</a>>
                wrote:</div>
            </div>
            <div><br>
            </div>
            <blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE"
              style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5;
              MARGIN:0 0 0 5;">
              <div>
                <div>
                  <div dir="ltr">Yup.. move along, nothing to see here.
                    <div><br>
                    </div>
                    <div>Once an outage is fixed, those who dwell on the
                      cause that they can do nothing about, are focusing
                      in the wrong place.</div>
                    <div><br>
                    </div>
                    <div>If your only transit was through a single
                      upstream, that is where you should be focusing,
                      not the provider.</div>
                  </div>
                  <div class="gmail_extra"><br clear="all">
                    <div>
                      <div dir="ltr">
                        <div><br>
                          ...Skeeve</div>
                        <div><br>
                        </div>
                        <div>
                          <div><b
                              style="font-size:13px;font-family:Calibri">Skeeve
                              Stevens - </b><span
                              style="font-size:13px;font-family:Calibri">eintellego
                              Networks Pty Ltd</span></div>
                          <div>
                            <div><span
                                style="font-family:Calibri;font-size:13px"><a
                                  moz-do-not-send="true"
                                  href="mailto:skeeve@eintellegonetworks.com"
                                  target="_blank">skeeve@eintellegonetworks.com</a> ; <a
                                  moz-do-not-send="true"
                                  href="http://www.eintellegonetworks.com/"
                                  target="_blank">www.eintellegonetworks.com</a></span><font>
                                <p
                                  style="font-family:Calibri;font-size:13px;margin:0px">Phone:
                                  1300 239 038; Cell +61 (0)414 753 383
                                  ; <a moz-do-not-send="true">skype://skeeve</a></p>
                                <p
                                  style="font-family:Calibri;font-size:13px;margin:0px"><a
                                    moz-do-not-send="true"
                                    href="http://facebook.com/eintellegonetworks"
                                    target="_blank">facebook.com/eintellegonetworks</a> ; <a
                                    moz-do-not-send="true"
                                    href="http://linkedin.com/in/skeeve"
                                    target="_blank">linkedin.com/in/skeeve</a> </p>
                                <p
                                  style="font-family:Calibri;font-size:13px;margin:0px"><a
                                    moz-do-not-send="true"
                                    href="http://twitter.com/theispguy"
                                    target="_blank">twitter.com/theispguy</a><span
                                    style="color:rgb(0,0,0)"> ; blog: </span><a
                                    moz-do-not-send="true"
                                    href="http://www.theispguy.com/"
                                    target="_blank">www.theispguy.com</a><br>
                                </p>
                                <p
                                  style="font-family:Calibri;font-size:13px;margin:0px"><img
                                    moz-do-not-send="true"
                                    src="http://eintellegonetworks.com/logos/ein09.png"><br>
                                </p>
                                <p style="margin:0px"><span
                                    style="color: rgb(127, 0, 127);
                                    font-family: Calibri, sans-serif;
                                    font-size: 13px;">The Experts Who
                                    The Experts Call</span></p>
                              </font></div>
                            <div
style="font-family:Calibri,sans-serif;font-size:14px;color:rgb(127,0,127)"><span
style="color:rgb(0,32,96);font-size:13px">Juniper - Cisco </span><span
                                style="color:rgb(0,32,96);font-size:13px">-
                                Cloud</span><span
                                style="color:rgb(0,32,96);font-size:13px"> </span><span
style="color:rgb(0,32,96);font-size:13px">- Consulting</span><span
                                style="color:rgb(0,32,96);font-size:13px"> </span><span
style="color:rgb(0,32,96);font-size:13px">- IPv4 Brokering</span></div>
                          </div>
                        </div>
                      </div>
                    </div>
                    <br>
                    <br>
                    <div class="gmail_quote">On 26 August 2014 23:19,
                      Kristoffer Sheather @ CloudCentral
                      <span dir="ltr"><<a moz-do-not-send="true"
                          href="mailto:kristoffer.sheather@cloudcentral.com.au"
                          target="_blank">kristoffer.sheather@cloudcentral.com.au</a>></span>
                      wrote:<br>
                      <blockquote class="gmail_quote" style="margin:0 0
                        0 .8ex;border-left:1px #ccc
                        solid;padding-left:1ex">
                        <span style="font-family: Arial, Helvetica,
                          sans-serif; font-size: 12px;">
                          <div>Shit broke, they fixed.</div>
                          <div> </div>
                          <div><EOM /></div>
                          <div> </div>
                          <hr align="center" size="2" width="100%">
                          <div><span style="font-family: tahoma, arial,
                              sans-serif; font-size: 10pt;"><b>From</b>:
                              "Wolfgang Nagele (AusRegistry)" <<a
                                moz-do-not-send="true"
                                href="mailto:wolfgang.nagele@ausregistry.com.au"
                                target="_blank">wolfgang.nagele@ausregistry.com.au</a>><br>
                              <b>Sent</b>: Tuesday, August 26, 2014
                              11:17 PM<br>
                              <b>To</b>: "James Spenceley" <<a
                                moz-do-not-send="true"
                                href="mailto:james@iroute.org"
                                target="_blank">james@iroute.org</a>><br>
                              <b>Cc</b>: "<a moz-do-not-send="true"
                                href="mailto:Ausnog@ausnog.net"
                                target="_blank">Ausnog@ausnog.net</a>"
                              <<a moz-do-not-send="true"
                                href="mailto:ausnog@ausnog.net"
                                target="_blank">ausnog@ausnog.net</a>><br>
                              <b>Subject</b>: SPAM-MED: Re: [AusNOG]
                              Vocus international service outage</span>
                            <div> </div>
                            <div>Hi James,</div>
                            <div> </div>
                            <div>Still waiting for the RfO on this whole
                              thing with the details. Neither seen one
                              here nor as a follow-up to customer
                              notifications.</div>
                            <div> </div>
                            <div>Cheers,</div>
                            <div>Wolfgang</div>
                            <div> </div>
                            <span></span>
                            <div>
                              <div><span>On 8/24/14, 12:38 AM, "Wolfgang
                                  Nagele (AusRegistry)" <<a
                                    moz-do-not-send="true"
                                    href="mailto:wolfgang.nagele@ausregistry.com.au"
                                    target="_blank">wolfgang.nagele@ausregistry.com.au</a>>
                                  wrote:</span></div>
                            </div>
                            <span></span>
                            <div> </div>
                            <span></span>
                            <blockquote style="BORDER-LEFT:#b5c4df 5
                              solid;PADDING:0 0 0 5;MARGIN:0 0 0 5">
                              <div>
                                <div
style="word-wrap:break-word;color:rgb(0,0,0);font-size:15px;font-family:Calibri,sans-serif">
                                  <div><span>Hi James,</span></div>
                                  <div> </div>
                                  <div><span>Hmm - can understand that
                                      but would have expected that there
                                      is sufficient redundancy in the LA
                                      landing of your network. I would
                                      have expected that a re-route and
                                      taking SJC largely out of the
                                      equation would be possible.
                                      Surprised to say the least …</span></div>
                                  <div> </div>
                                  <div><span>Cheers,</span></div>
                                  <div><span>Wolfgang</span></div>
                                  <div> </div>
                                  <div>
                                    <div><span><span>On 8/24/14, 12:10
                                          AM, "James Spenceley" <<a
                                            moz-do-not-send="true"
                                            href="mailto:james@iroute.org"
                                            target="_blank">james@iroute.org</a>>
                                          wrote:</span></span></div>
                                  </div>
                                  <div> </div>
                                  <blockquote style="BORDER-LEFT:#b5c4df
                                    5 solid;PADDING:0 0 0 5;MARGIN:0 0 0
                                    5">
                                    <div>
                                      <div dir="auto">
                                        <div><span><span>Early mail is a
                                              power surge in a US DC has
                                              damaged both core routers.
                                              Wouldn't surprise me if
                                              transport from other
                                              providers out of that
                                              building will be having
                                              similar issues. </span></span></div>
                                        <div> </div>
                                        <div><span><span>Circuits are
                                              being moved directly to
                                              borders as we speak.</span></span></div>
                                        <div> </div>
                                        <div><br>
                                          <br>
                                          <span><span>Sent from my
                                              iPhone</span></span></div>
                                        <div><br>
                                          <span><span>On 24 Aug 2014, at
                                              0:00, Jared Hirst <<a
                                                moz-do-not-send="true"
                                                href="mailto:jared.hirst@serversaustralia.com.au"
                                                target="_blank">jared.hirst@serversaustralia.com.au</a>>
                                              wrote:</span></span><br>
                                           </div>
                                        <blockquote type="cite">
                                          <div>
                                            <div dir="ltr">
                                              <div class="gmail_default"
                                                style="font-family:'trebuchet
ms',sans-serif;color:rgb(0,0,0)">
                                                <span><span>WOW.... It
                                                    has taken 2 hours to
                                                    get remote hands to
                                                    the DC with what
                                                    seems to be a device
                                                    with no redundancy?</span></span></div>
                                              <div class="gmail_default"
                                                style="font-family:'trebuchet
ms',sans-serif;color:rgb(0,0,0)">
                                                 </div>
                                              <div class="gmail_default"
                                                style="font-family:'trebuchet
ms',sans-serif;color:rgb(0,0,0)">
                                                <table
style="font-family:verdana,geneva,sans-serif;margin:auto;color:rgb(51,51,51);font-size:12px;border-collapse:collapse">
                                                  <tbody>
                                                    <tr>
                                                      <td
                                                        style="padding:5px"><span><span>2014/08/23
                                                          13:55<br>
                                                          UTC</span></span></td>
                                                      <td
                                                        style="padding:5px">
                                                        <p><span><span>Engineers
                                                          are currently
                                                          awaiting
                                                          remote support
                                                          in the US.
                                                          Links will be
                                                          physically
                                                          moved from the
                                                          failed device
                                                          in order to
                                                          restore
                                                          services on an
                                                          alternate
                                                          device.</span></span></p>
                                                      </td>
                                                    </tr>
                                                  </tbody>
                                                </table>
                                              </div>
                                              <div class="gmail_extra"> 
                                                <div class="gmail_quote"><span><span>On
                                                      Sat, Aug 23, 2014
                                                      at 11:29 PM,
                                                      Andrew Yager
                                                      <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:andrew@rwts.com.au" target="_blank">andrew@rwts.com.au</a>></span>
                                                      wrote:</span></span>
                                                  <blockquote
                                                    class="gmail_quote"
                                                    style="margin:0 0 0
                                                    .8ex;border-left:1px
                                                    #ccc
                                                    solid;padding-left:1ex">
                                                    <div dir="ltr">
                                                      <div><span><span>[hijacking
                                                          the thread…]</span></span></div>
                                                      <div> </div>
                                                      <span><span>They
                                                          say there is a
                                                          big rewrite
                                                          coming on the
                                                          way rpd and
                                                          sampled
                                                          interact in
                                                          14.2; and the
                                                          slow
                                                          convergance
                                                          issues have
                                                          been fixed in
                                                          more releases
                                                          than I care to
                                                          remember right
                                                          now… but
                                                          people say
                                                          they are
                                                          pretty good in
                                                          the 12.3r6
                                                          train. Our
                                                          MX80's are
                                                          slated for
                                                          upgrade to
                                                          that at some
                                                          stage in the
                                                          next few
                                                          months. </span>
                                                      </span>
                                                      <div> </div>
                                                      <span><span></span></span>
                                                      <div><span><span>Some
                                                          noise about
                                                          this on j-nsp
                                                          again today.</span></span>
                                                        <div> </div>
                                                        <span><span><span></span></span></span>
                                                        <div><span><span><span><font
color="#888888">Andrew</font></span></span></span>
                                                          <div>
                                                          <div> </div>
                                                          <div> </div>
                                                          </div>
                                                          <span><span><span></span></span></span></div>
                                                        <span><span><span></span></span></span></div>
                                                      <span><span></span></span></div>
                                                    <div>
                                                      <div>
                                                        <div
                                                          class="gmail_extra"> 
                                                          <div
                                                          class="gmail_quote"><span><span>On
                                                          23 August 2014
                                                          23:23,
                                                          Jonathan
                                                          Thorpe <span
                                                          dir="ltr">
                                                          <<a
                                                          moz-do-not-send="true"
href="mailto:jthorpe@conexim.com.au" target="_blank">jthorpe@conexim.com.au</a>></span>
                                                          wrote:</span></span>
                                                          <blockquote
                                                          class="gmail_quote"
                                                          style="margin:0
                                                          0 0
                                                          .8ex;border-left:1px
                                                          #ccc
                                                          solid;padding-left:1ex">
                                                          <div
                                                          link="blue"
                                                          vlink="purple"
                                                          lang="EN-GB">
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);">True,
                                                          but they
                                                          otherwise work
                                                          exceptionally
                                                          well.</span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"> </span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);">I’m
                                                          not sure what
                                                          kind of
                                                          PowerPC CPU is
                                                          doing all the
                                                          work on an
                                                          MX80’s RE, but
                                                          I do sometimes
                                                          wonder if the
                                                          CPU on a
                                                          <$40
                                                          Raspberry Pi
                                                          might be more
                                                          up to the job
                                                          :-P</span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"> </span></span></span></p>
                                                          <div>
                                                          <div
                                                          style="border:none;border-top:solid
                                                          #e1e1e1
                                                          1.0pt;padding:3.0pt
                                                          0cm 0cm 0cm">
                                                          <p
                                                          class="MsoNormal"><span><span><b><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">From:</span></b><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">
                                                          Tony Wicks
                                                          [mailto:<a
                                                          moz-do-not-send="true"
href="mailto:tony@wicks.co.nz" target="_blank">tony@wicks.co.nz</a>]<br>
                                                          <b>Sent:</b>
                                                          Saturday, 23
                                                          August 2014
                                                          11:13 PM<br>
                                                          <b>To:</b>
                                                          Jonathan
                                                          Thorpe<br>
                                                          <b>Cc:</b> '<a
moz-do-not-send="true" href="mailto:Ausnog@ausnog.net" target="_blank">Ausnog@ausnog.net</a>'<br>
                                                          <b>Subject:</b>
                                                          RE: [AusNOG]
                                                          Vocus
                                                          international
                                                          service outage</span></span></span></p>
                                                          </div>
                                                          </div>
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span> </span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-NZ">Well,
                                                          if you buy the
                                                          big chassis
                                                          boxes…..</span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-NZ"> </span></span></span></p>
                                                          <div>
                                                          <div
                                                          style="border:none;border-top:solid
                                                          #e1e1e1
                                                          1.0pt;padding:3.0pt
                                                          0cm 0cm 0cm">
                                                          <p
                                                          class="MsoNormal"><span><span><b><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">From:</span></b><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">
                                                          AusNOG [</span><a
moz-do-not-send="true" href="mailto:ausnog-bounces@lists.ausnog.net"
                                                          target="_blank"><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">mailto:ausnog-bounces@lists.ausnog.net</span></a><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">]
                                                          <b>On Behalf
                                                          Of </b>Jonathan
                                                          Thorpe<br>
                                                          <b>Sent:</b>
                                                          Sunday, 24
                                                          August 2014
                                                          1:10 a.m.<br>
                                                          <b>To:</b>
                                                          Andrew Yager;
                                                          Jared Hirst<br>
                                                          <b>Cc:</b> </span><a
moz-do-not-send="true" href="mailto:Ausnog@ausnog.net" target="_blank"><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">Ausnog@ausnog.net</span></a><br>
                                                          <span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US"><b>Subject:</b>
                                                          Re: [AusNOG]
                                                          Vocus
                                                          international
                                                          service outage</span></span></span></p>
                                                          </div>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          lang="EN-NZ"> </span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);">Glad
                                                          I’m not the
                                                          only one
                                                          holding my
                                                          breath on our
                                                          MXs :)</span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"> </span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><b><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">From:</span></b><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">
                                                          AusNOG [</span><a
moz-do-not-send="true" href="mailto:ausnog-bounces@lists.ausnog.net"
                                                          target="_blank"><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">mailto:ausnog-bounces@lists.ausnog.net</span></a><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">]
                                                          <b>On Behalf
                                                          Of </b>Andrew
                                                          Yager<br>
                                                          <b>Sent:</b>
                                                          Saturday, 23
                                                          August 2014
                                                          10:55 PM<br>
                                                          <b>To:</b>
                                                          Jared Hirst<br>
                                                          <b>Cc:</b> </span><a
moz-do-not-send="true" href="mailto:Ausnog@ausnog.net" target="_blank"><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">Ausnog@ausnog.net</span></a><br>
                                                          <span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US"><b>Subject:</b>
                                                          Re: [AusNOG]
                                                          Vocus
                                                          international
                                                          service outage</span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span> </span></span></p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span>We've
                                                          done the same
                                                          (about 20
                                                          minutes ago).</span></span></p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span> </span></span></p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span>Right
                                                          now I hate how
                                                          long Juniper
                                                          MX's take to
                                                          stabilise
                                                          their routing
                                                          table with
                                                          sampling on.</span></span></p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span> </span></span></p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span>Andrew</span></span></p>
                                                          </div>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"
style="margin-bottom:12.0pt"><span><span> </span></span></p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span>On
                                                          23 August 2014
                                                          22:41, Jared
                                                          Hirst <<a
                                                          moz-do-not-send="true"
href="mailto:jared.hirst@serversaustralia.com.au" target="_blank">jared.hirst@serversaustralia.com.au</a>>
                                                          wrote:</span></span></p>
                                                          <blockquote
                                                          style="border:none;border-left:solid
                                                          #cccccc
                                                          1.0pt;padding:0cm
                                                          0cm 0cm
6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-family:
                                                          'Trebuchet
                                                          MS',
                                                          sans-serif;
                                                          color: black;">We
                                                          have just
                                                          turned Vocus
                                                          off. Using
                                                          other
                                                          providers for
                                                          now, as the
                                                          flapping is
                                                          causing it to
                                                          go up and
                                                          down.</span></span></span></p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"
style="margin-bottom:12.0pt"><span><span> </span></span></p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span>On
                                                          Sat, Aug 23,
                                                          2014 at 10:37
                                                          PM, Daniel
                                                          Watson <<a
moz-do-not-send="true" href="mailto:Daniel@glovine.com.au"
                                                          target="_blank">Daniel@glovine.com.au</a>>
                                                          wrote:</span></span></p>
                                                          <blockquote
                                                          style="border:none;border-left:solid
                                                          #cccccc
                                                          1.0pt;padding:0cm
                                                          0cm 0cm
6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU">Indeed
                                                          seeing some
                                                          big drops in
                                                          gaming traffic
                                                          at present,
                                                          normally we
                                                          see above
                                                          60mbit on
                                                          weekends at
                                                          evenings, but
                                                          not even
                                                          seeing 40mbit
                                                          at present :S</span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU"> </span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU"> </span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU">Regards,</span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU">Daniel
                                                          Watson</span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU">Network
                                                          Administrator
                                                          / Network
                                                          Operations
                                                          Manager</span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU"> </span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU">E
                                                          </span><a
                                                          moz-do-not-send="true"
href="mailto:Daniel@GloVine.com.au" target="_blank"><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-AU">Daniel@GloVine.com.au</span></a></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU">W
                                                          </span><a
                                                          moz-do-not-send="true"
href="http://www.GloVine.com.au" target="_blank"><span style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-AU">www.GloVine.com.au</span></a></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;
                                                          color: rgb(31,
                                                          73, 125);"
                                                          lang="EN-AU"> </span></span></span></p>
                                                          <p
                                                          class="MsoNormal"><span><span><b><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">From:</span></b><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">
                                                          AusNOG
                                                          [mailto:</span><a
moz-do-not-send="true" href="mailto:ausnog-bounces@lists.ausnog.net"
                                                          target="_blank"><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">ausnog-bounces@lists.ausnog.net</span></a><span
                                                          style="font-size:
                                                          11pt;
                                                          font-family:
                                                          Calibri,
                                                          sans-serif;"
                                                          lang="EN-US">]
                                                          <b>On Behalf
                                                          Of </b>Jared
                                                          Hirst<br>
                                                          <b>Sent:</b>
                                                          Saturday, 23
                                                          August 2014
                                                          10:35 PM</span></span></span></p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><br>
                                                          <span><span><span
                                                          lang="EN-AU"><b>To:</b>
                                                          Andrew Cox<br>
                                                          <b>Cc:</b> </span><a
moz-do-not-send="true" href="mailto:Ausnog@ausnog.net" target="_blank"><span
                                                          lang="EN-AU">Ausnog@ausnog.net</span></a><br>
                                                          <span
                                                          lang="EN-AU"><b>Subject:</b>
                                                          Re: [AusNOG]
                                                          Vocus
                                                          international
                                                          service outage</span></span></span></p>
                                                          </div>
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          lang="EN-AU"> </span></span></span></p>
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          style="font-family:
                                                          'Trebuchet
                                                          MS',
                                                          sans-serif;
                                                          color: black;"
                                                          lang="EN-AU">Yeah
                                                          we are seeing
                                                          this!
                                                          Everything
                                                          running via
                                                          them is
                                                          flapping, they
                                                          claim to have
                                                          'routed around
                                                          it' but that's
                                                          not the case.
                                                          Very
                                                          frustrating as
                                                          it's been an
                                                          hour and no
                                                          one there
                                                          seems to know
                                                          whats going
                                                          on....</span></span></span></p>
                                                          </div>
                                                          <div>
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"
style="margin-bottom:12.0pt"><span><span><span lang="EN-AU"> </span></span></span></p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          lang="EN-AU">On
                                                          Sat, Aug 23,
                                                          2014 at 10:29
                                                          PM, Andrew Cox
                                                          <</span><a
moz-do-not-send="true" href="mailto:andrew.cox@bigair.net.au"
                                                          target="_blank"><span
                                                          lang="EN-AU">andrew.cox@bigair.net.au</span></a><span
                                                          lang="EN-AU">>
                                                          wrote:</span></span></span></p>
                                                          <blockquote
                                                          style="border:none;border-left:solid
                                                          #cccccc
                                                          1.0pt;padding:0cm
                                                          0cm 0cm
6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          lang="EN-AU">Hey
                                                          All,<br>
                                                          <br>
                                                          Just saw the
                                                          dashboards
                                                          light up with
                                                          connectivity
                                                          issues
                                                          internationally
                                                          for Vocus
                                                          services and
                                                          thought I'd
                                                          make others
                                                          aware.<br>
                                                          <br>
                                                          Vocus outage
                                                          report is
                                                          saying: "core
                                                          network link
                                                          between 59
                                                          Doody Street,
                                                          Alexandria and
                                                          55 South
                                                          Market Street,
                                                          San Jose has
                                                          failed" which
                                                          hopefully
                                                          isn't a
                                                          Southern Cross
                                                          fault!<br>
                                                          <br>
                                                          Anyone else
                                                          seeing this or
                                                          have more
                                                          info?</span></span></span></p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          lang="EN-AU"> </span></span></span></p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          lang="EN-AU">Cheers,</span></span></span></p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"
style="margin-bottom:12.0pt"><span><span><span lang="EN-AU">Andrew</span></span></span></p>
                                                          </div>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"
style="margin-bottom:12.0pt"><br>
                                                          <span><span><span
                                                          lang="EN-AU">_______________________________________________<br>
                                                          AusNOG mailing
                                                          list</span><br>
                                                          <a
                                                          moz-do-not-send="true"
href="mailto:AusNOG@lists.ausnog.net" target="_blank"><span lang="EN-AU">AusNOG@lists.ausnog.net</span></a><br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank"><span
                                                          lang="EN-AU">http://lists.ausnog.net/mailman/listinfo/ausnog</span></a></span></span></p>
                                                          </blockquote>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"><br>
                                                           </p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          lang="EN-AU"> </span></span></span></p>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          lang="EN-AU">--
                                                          </span></span></span></p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span><span
                                                          lang="EN-AU"> </span></span></span></p>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </blockquote>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"
style="margin-bottom:12.0pt"><span><span> </span></span></p>
                                                          </div>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"
style="margin-bottom:12.0pt"><br>
                                                          <span><span>_______________________________________________<br>
                                                          AusNOG mailing
                                                          list<br>
                                                          <a
                                                          moz-do-not-send="true"
href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a><br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a></span></span></p>
                                                          </blockquote>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"><br>
                                                           </p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><span><span> </span></span></p>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"><span><span>--<br>
                                                          <b>Andrew
                                                          Yager,
                                                          Managing
                                                          Director</b>  
                                                          <i>MACS (Snr)
                                                          CP BCompSc MCP</i><br>
                                                          Real World
                                                          Technology
                                                          Solutions Pty
                                                          Ltd - IT
                                                          people you can
                                                          trust<br>
                                                          ph: <a
                                                          moz-do-not-send="true"
href="tel:1300%20798%20718" value="+611300798718" target="_blank">1300
                                                          798 718</a> or
                                                          <a
                                                          moz-do-not-send="true"
href="tel:%2802%29%209037%200500" value="+61290370500" target="_blank">(02)
                                                          9037 0500</a><br>
                                                          fax: <a
                                                          moz-do-not-send="true"
href="tel:%2802%29%209037%200591" value="+61290370591" target="_blank">(02)
                                                          9037 0591</a><br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://www.rwts.com.au/" target="_blank">http://www.rwts.com.au/</a></span></span></p>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          <br>
                                                          <span><span>_______________________________________________<br>
                                                          AusNOG mailing
                                                          list<br>
                                                          <a
                                                          moz-do-not-send="true"
href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a><br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a></span></span><br>
                                                           </blockquote>
                                                          <span><span></span></span></div>
                                                           
                                                          <div> </div>
                                                          <span><span>--<br>
                                                          <b>Andrew
                                                          Yager,
                                                          Managing
                                                          Director</b>  
                                                          <i>MACS (Snr)
                                                          CP BCompSc MCP</i><br>
                                                          Real World
                                                          Technology
                                                          Solutions Pty
                                                          Ltd - IT
                                                          people you can
                                                          trust<br>
                                                          ph: <a
                                                          moz-do-not-send="true"
href="tel:1300%20798%20718" value="+611300798718" target="_blank">1300
                                                          798 718</a> or
                                                          <a
                                                          moz-do-not-send="true"
href="tel:%2802%29%209037%200500" value="+61290370500" target="_blank">(02)
                                                          9037 0500</a><br>
                                                          fax: <a
                                                          moz-do-not-send="true"
href="tel:%2802%29%209037%200591" value="+61290370591" target="_blank">(02)
                                                          9037 0591</a><br>
                                                          <a
                                                          moz-do-not-send="true"
href="http://www.rwts.com.au/" target="_blank">http://www.rwts.com.au/</a></span></span></div>
                                                      </div>
                                                    </div>
                                                    <br>
                                                    <span><span>_______________________________________________<br>
                                                        AusNOG mailing
                                                        list<br>
                                                        <a
                                                          moz-do-not-send="true"
href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a><br>
                                                        <a
                                                          moz-do-not-send="true"
href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a></span></span><br>
                                                     </blockquote>
                                                  <span><span></span></span></div>
                                                 
                                                <div> </div>
                                                 
                                                <div dir="ltr"> </div>
                                              </div>
                                            </div>
                                          </div>
                                        </blockquote>
                                        <blockquote type="cite">
                                          <div><span><span><span>_______________________________________________</span><br>
                                                <span>AusNOG mailing
                                                  list</span><br>
                                                <span><a
                                                    moz-do-not-send="true"
href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a></span><br>
                                                <span><a
                                                    moz-do-not-send="true"
href="http://lists.ausnog.net/mailman/listinfo/ausnog" target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a></span></span></span></div>
                                        </blockquote>
                                      </div>
                                    </div>
                                  </blockquote>
                                </div>
                              </div>
                            </blockquote>
                            <span></span></div>
                        </span><br>
                        _______________________________________________<br>
                        AusNOG mailing list<br>
                        <a moz-do-not-send="true"
                          href="mailto:AusNOG@lists.ausnog.net">AusNOG@lists.ausnog.net</a><br>
                        <a moz-do-not-send="true"
                          href="http://lists.ausnog.net/mailman/listinfo/ausnog"
                          target="_blank">http://lists.ausnog.net/mailman/listinfo/ausnog</a><br>
                        <br>
                      </blockquote>
                    </div>
                    <br>
                  </div>
                </div>
              </div>
            </blockquote>
          </span>
        </div>
      </blockquote>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
AusNOG mailing list
<a class="moz-txt-link-abbreviated" href="mailto:AusNOG@lists.ausnog.net">AusNOG@lists.ausnog.net</a>
<a class="moz-txt-link-freetext" href="http://lists.ausnog.net/mailman/listinfo/ausnog">http://lists.ausnog.net/mailman/listinfo/ausnog</a>
</pre>
    </blockquote>
    <br>
    <br>
    <pre class="moz-signature" cols="72">-- 
/* Matt Perkins 
        Direct 1300 137 379     Spectrum Networks Ptd. Ltd. 
        Office 1300 133 299     <a class="moz-txt-link-abbreviated" href="mailto:matt@spectrum.com.au">matt@spectrum.com.au</a> 
                                Level 6, 350 George Street Sydney 2000
        PGP/GNUPG Public Key can be found at  <a class="moz-txt-link-freetext" href="http://pgp.mit.edu">http://pgp.mit.edu</a> 
*/
</pre>
  </body>
</html>