[AusNOG] Vocus international service outage

Daniel Watson Daniel at glovine.com.au
Sun Aug 24 15:45:19 EST 2014


Stupid moderation :@

Apologies, these were sent lastnight during the outage, and we had big domestic issues last night, but all fixed now :D


Regards,
Daniel Watson
Network Administrator / Network Operations Manager

E Daniel at GloVine.com.au
W www.GloVine.com.au

From: Damien Gardner Jnr [mailto:rendrag at rendrag.net]
Sent: Sunday, 24 August 2014 3:42 PM
To: Daniel Watson
Cc: Andrew Cox; Jared Hirst; Matt Taylor; Ausnog at ausnog.net
Subject: Re: [AusNOG] Vocus international service outage

Looks good on telstra.

damien at toolbox:~$ traceroute 103.18.207.24
traceroute to 103.18.207.24 (103.18.207.24), 30 hops max, 60 byte packets
 1  rtr01-vlan101.wyong01.nsw.rendrag.net.au<http://rtr01-vlan101.wyong01.nsw.rendrag.net.au> (203.42.121.161)  4.852 ms  5.170 ms  5.580 ms
 2  gw130.ken10.sydney.telstra.net<http://gw130.ken10.sydney.telstra.net> (203.45.254.1)  27.130 ms  28.008 ms  28.448 ms
 3  tengigabitethernet7-1.ult2.sydney.telstra.net<http://tengigabitethernet7-1.ult2.sydney.telstra.net> (203.50.20.51)  30.927 ms  31.323 ms  31.311 ms
 4  vocusf.lnk.telstra.net<http://vocusf.lnk.telstra.net> (139.130.73.106)  33.533 ms  33.951 ms  34.054 ms
 5  ten-0-0-0-1.cor03.syd03.nsw.VOCUS.net.au<http://ten-0-0-0-1.cor03.syd03.nsw.VOCUS.net.au> (175.45.72.72)  34.961 ms  36.609 ms  36.945 ms
 6  ten-0-1-0.bdr02.syd03.nsw.VOCUS.net.au<http://ten-0-1-0.bdr02.syd03.nsw.VOCUS.net.au> (114.31.192.61)  38.232 ms  18.997 ms  19.417 ms
 7  as45671.cust.bdr02.syd03.nsw.VOCUS.net.au<http://as45671.cust.bdr02.syd03.nsw.VOCUS.net.au> (114.31.200.126)  21.353 ms  21.467 ms  25.016 ms
 8  bdr-1.nsw.au.glonetwork.com.au<http://bdr-1.nsw.au.glonetwork.com.au> (27.50.72.99)  25.261 ms  25.948 ms  26.251 ms
 9  103.18.207.24 (103.18.207.24)  29.817 ms  32.019 ms  32.166 ms

Also fine on SAU ;)

damien at echelon:~$ traceroute 103.18.207.24
traceroute to 103.18.207.24 (103.18.207.24), 30 hops max, 60 byte packets
 1  bdr01-vlan49.syd01.nsw.rendrag.net.au<http://bdr01-vlan49.syd01.nsw.rendrag.net.au> (103.235.52.49)  0.501 ms  0.472 ms  0.305 ms
 2  core-bdr-1-1-ve309.sau.net.au<http://core-bdr-1-1-ve309.sau.net.au> (27.50.65.210)  0.654 ms  0.583 ms  0.594 ms
 3  202.130.32.77 (202.130.32.77)  0.605 ms  0.688 ms  0.792 ms
 4  bdr-1.nsw.au.glonetwork.com.au<http://bdr-1.nsw.au.glonetwork.com.au> (27.50.72.99)  9.143 ms  9.133 ms  9.113 ms
 5  103.18.207.24 (103.18.207.24)  11.992 ms  12.484 ms  12.470 ms

Route via Eftel is a little hinkey, but works..

 # ADDRESS                          LOSS SENT    LAST     AVG    BEST   WORST STD-DEV STATUS                          1 vlan151.bdr01.syd01.eftel.com<http://vlan151.bdr01.syd01.eftel.com>      0%    4   0.9ms       1     0.9     1.2     0.1                                                          2 ge0-1-902.lns01.per01.eftel.com<http://ge0-1-902.lns01.per01.eftel.com>    0%    4  59.6ms    59.5    59.3    59.6     0.1                                                      3 vocus.ix.waia.asn.au<http://vocus.ix.waia.asn.au>               0%    4    61ms    59.9    59.3      61     0.7                                                            4 ge-0-0-0-18.cor01.per02.wa.vo...   0%    4    56ms    56.1    55.9    56.6     0.3                                                      5 ten-0-2-0-7.cor03.syd03.nsw.v...   0%    4  56.7ms    56.5    56.2    56.7     0.2                                                      6 ten-0-1-0.bdr02.syd03.nsw.voc...   0%    4  55.7ms    55.6    55.6    55.7       0                                                      7 as45671.cust.bdr02.syd03.nsw....   0%    3  55.8ms    56.1    55.7    56.8     0.5                                                    8 bdr-1.nsw.au.glonetwork.com.au<http://bdr-1.nsw.au.glonetwork.com.au>   66..    3 timeout    57.4    57.4    57.4       0                                                        9 103.18.207.24                      0%    3  60.4ms    61.1    59.4    63.6     1.8

On 23 August 2014 23:01, Daniel Watson <Daniel at glovine.com.au<mailto:Daniel at glovine.com.au>> wrote:
Just curious if anybody else is seeing any other issues with other providers?  TPG, AAPT or EQX?


C:\Users\danie_000>tracert 203.26.189.240

Tracing route to 203.26.189.240 over a maximum of 30 hops

  1    14 ms    16 ms     4 ms  192.168.1.1
  2    29 ms    27 ms    35 ms  10.20.21.170
  3    30 ms    40 ms    81 ms  nme-sot-dry-crt1-Te0-3-0-14.tpgi.com.au<http://nme-sot-dry-crt1-Te0-3-0-14.tpgi.com.au> [203.26.22.225]
  4    28 ms    31 ms    27 ms  203.8.183.21
  5    38 ms    38 ms    40 ms  po9.mflindist02.aapt.net.au<http://po9.mflindist02.aapt.net.au> [202.10.14.32]
  6    37 ms    42 ms    39 ms  te2-1-110.mflindist01.aapt.net.au<http://te2-1-110.mflindist01.aapt.net.au> [202.10.12.136]
  7     *       37 ms    37 ms  te0-3-4-0.mflincore01.aapt.net.au<http://te0-3-4-0.mflincore01.aapt.net.au> [202.10.12.11]
  8    48 ms    38 ms    37 ms  bu2.sglebcore01.aapt.net.au<http://bu2.sglebcore01.aapt.net.au> [202.10.10.72]
  9    42 ms    43 ms    41 ms  po11.sglebdist01.aapt.net.au<http://po11.sglebdist01.aapt.net.au> [202.10.12.6]
10    46 ms    39 ms    36 ms  1-1-1.sglebinte01.aapt.net.au<http://1-1-1.sglebinte01.aapt.net.au> [203.131.60.39]
11    39 ms    45 ms    42 ms  203-174-189-250.syd.static-ipl.aapt.com.au<http://203-174-189-250.syd.static-ipl.aapt.com.au> [203.
174.189.250]
12    41 ms    41 ms    38 ms  xe-0-1-1.gw102.sy1.ap.equinix.com<http://xe-0-1-1.gw102.sy1.ap.equinix.com> [27.111.240.146]
13    49 ms    40 ms    44 ms  bdr-1-1-te2-49.sau.net.au<http://bdr-1-1-te2-49.sau.net.au> [203.190.232.234]
14    37 ms    41 ms    37 ms  202.130.32.77
15    40 ms    49 ms    44 ms  bdr-1.nsw.au.glonetwork.com.au<http://bdr-1.nsw.au.glonetwork.com.au> [27.50.72.99]
16    41 ms    43 ms    40 ms  bdr-1.nsw.au.dedicatedhosting-solutions.com.au<http://bdr-1.nsw.au.dedicatedhosting-solutions.com.au> [
103.18.205.241]
17    53 ms    46 ms    46 ms  203.26.189.240

Trace complete.

Above works,

C:\Users\danie_000>tracert 103.18.207.24

Tracing route to 103.18.207.24 over a maximum of 30 hops

  1     5 ms     4 ms     4 ms  192.168.1.1
  2    28 ms    34 ms    28 ms  10.20.21.170
  3    27 ms    26 ms    37 ms  nme-sot-dry-agr2-Te2-6.tpgi.com.au<http://nme-sot-dry-agr2-Te2-6.tpgi.com.au> [202.7.173.69]
  4    32 ms    35 ms    30 ms  nme-sot-dry-crt2-Te0-1-0-4.tpgi.com.au<http://nme-sot-dry-crt2-Te0-1-0-4.tpgi.com.au> [203.26.22.49]
  5    45 ms    42 ms    38 ms  syd-sot-ken-crt1-te-gi-0-6-0-2.tpgi.com.au<http://syd-sot-ken-crt1-te-gi-0-6-0-2.tpgi.com.au> [203.219.106.189]
  6    40 ms    38 ms    38 ms  203-219-35-4.static.tpgi.com.au<http://203-219-35-4.static.tpgi.com.au> [203.219.35.4]
  7     *        *        *     Request timed out.
  8     *        *        *     Request timed out.
  9     *        *        *     Request timed out.
10     *        *        *     Request timed out.
11     *        *        *     Request timed out.
12     *        *        *     Request timed out.
13     *        *        *     Request timed out.
14     *        *        *     Request timed out.
15     *        *        *     Request timed out.
16     *        *        *     Request timed out.
17     *        *        *     Request timed out.
18     *        *        *     Request timed out.
19     *        *        *     Request timed out.
20     *        *        *     Request timed out.
21     *        *        *     Request timed out.
22     *        *        *     Request timed out.
23     *        *        *     Request timed out.
24     *        *        *     Request timed out.
25     *        *        *     Request timed out.
26  te0-1-0-3.ccr22.lax04.atlas.cogentco.com<http://te0-1-0-3.ccr22.lax04.atlas.cogentco.com> [38.104.210.53]  reports: Destinat
ion net unreachable.

Trace complete.

Above doesnt


Regards,
Daniel Watson
Network Administrator / Network Operations Manager

E Daniel at GloVine.com.au<mailto:Daniel at GloVine.com.au>
W www.GloVine.com.au<http://www.GloVine.com.au>

From: Andrew Cox [mailto:andrew.cox at bigair.net.au<mailto:andrew.cox at bigair.net.au>]
Sent: Saturday, 23 August 2014 10:56 PM
To: Jared Hirst
Cc: Daniel Watson; Ausnog at ausnog.net<mailto:Ausnog at ausnog.net>

Subject: Re: [AusNOG] Vocus international service outage

Likewise we've dropped Vocus entirely for now until we hear more.

It looked like although we weren't receiving any traffic, that they were still advertising our prefixes upstream.

- Andrew

On 23 August 2014 22:41, Jared Hirst <jared.hirst at serversaustralia.com.au<mailto:jared.hirst at serversaustralia.com.au>> wrote:
We have just turned Vocus off. Using other providers for now, as the flapping is causing it to go up and down.

On Sat, Aug 23, 2014 at 10:37 PM, Daniel Watson <Daniel at glovine.com.au<mailto:Daniel at glovine.com.au>> wrote:
Indeed seeing some big drops in gaming traffic at present, normally we see above 60mbit on weekends at evenings, but not even seeing 40mbit at present :S


Regards,
Daniel Watson
Network Administrator / Network Operations Manager

E Daniel at GloVine.com.au<mailto:Daniel at GloVine.com.au>
W www.GloVine.com.au<http://www.GloVine.com.au>

From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net<mailto:ausnog-bounces at lists.ausnog.net>] On Behalf Of Jared Hirst
Sent: Saturday, 23 August 2014 10:35 PM

To: Andrew Cox
Cc: Ausnog at ausnog.net<mailto:Ausnog at ausnog.net>
Subject: Re: [AusNOG] Vocus international service outage

Yeah we are seeing this! Everything running via them is flapping, they claim to have 'routed around it' but that's not the case. Very frustrating as it's been an hour and no one there seems to know whats going on....

On Sat, Aug 23, 2014 at 10:29 PM, Andrew Cox <andrew.cox at bigair.net.au<mailto:andrew.cox at bigair.net.au>> wrote:
Hey All,

Just saw the dashboards light up with connectivity issues internationally for Vocus services and thought I'd make others aware.

Vocus outage report is saying: "core network link between 59 Doody Street, Alexandria and 55 South Market Street, San Jose has failed" which hopefully isn't a Southern Cross fault!

Anyone else seeing this or have more info?

Cheers,
Andrew

_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog



--




_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog



--

Damien Gardner Jnr
VK2TDG. Dip EE. GradIEAust
rendrag at rendrag.net<mailto:rendrag at rendrag.net> -  http://www.rendrag.net/
--
We rode on the winds of the rising storm,
 We ran to the sounds of thunder.
We danced among the lightning bolts,
 and tore the world asunder
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20140824/5358da87/attachment-0001.html>


More information about the AusNOG mailing list