[AusNOG] Vocus international service outage
Trent Farrell
tfarrell at riotgames.com
Sun Aug 24 17:58:07 EST 2014
Sometimes, things break
On Sunday, August 24, 2014, Seamus Ryan <s.ryan at uber.com.au> wrote:
> I see a rather large upside to the happenings over the last 24 hours.
>
>
>
>
>
> At next AusNOG drinks next week, sponsored by who else but Vocus, things
> will be different. For myself and my fellow companions I envisage the
> following:
>
>
>
> · Upon arrival, an amazing show of lights, pyrotechnics and sound
> shall erupt.
>
> · A monkey dressed in a clown outfit shall roll up to all
> attendees on roller-skates, offering a selection of fine vodkas
>
> · The ground I walk on that evening will be drizzled with freshly
> plucked rose petals so my feet may feel the soft earth while not needing
> protection from the harsh elements.
>
> · James, sporting the sharpest tuxedo known to man, will kneel
> before us while offering delicious canapés ranging from the rarest caviar
> to exquisite cheeses.
>
> · A row of massage chairs equipped with drink holders shall line
> the bar.
>
> · The ambiance of the evening will reflect that of a private
> villa in the Maldives
>
> · Dan, dressed in a pink tutu for our entertainment, will drift
> around whispering in the ears of all attendees “You are an Angel, we love
> you for who you are”
>
> · At closure of the evening, a line of powerful sports cars
> ranging from R35’s, 458’s, C63’s, Zondas, Aventador’s and MP4-12C’s will
> wait patiently along King St Warf to take us safely home to our area of
> rest for the evening, be it hotel or home.
>
>
>
> Cheers,
>
> Seamus
>
>
>
>
>
>
>
> *From:* AusNOG [mailto:ausnog-bounces at lists.ausnog.net
> <javascript:_e(%7B%7D,'cvml','ausnog-bounces at lists.ausnog.net');>] *On
> Behalf Of *Damien Gardner Jnr
> *Sent:* Sunday, 24 August 2014 3:42 PM
> *To:* Daniel Watson
> *Cc:* Ausnog at ausnog.net
> <javascript:_e(%7B%7D,'cvml','Ausnog at ausnog.net');>; Matt Taylor
> *Subject:* Re: [AusNOG] Vocus international service outage
>
>
>
> Looks good on telstra.
>
>
>
> damien at toolbox:~$ traceroute 103.18.207.24
>
> traceroute to 103.18.207.24 (103.18.207.24), 30 hops max, 60 byte packets
>
> 1 rtr01-vlan101.wyong01.nsw.rendrag.net.au (203.42.121.161) 4.852 ms
> 5.170 ms 5.580 ms
>
> 2 gw130.ken10.sydney.telstra.net (203.45.254.1) 27.130 ms 28.008 ms
> 28.448 ms
>
> 3 tengigabitethernet7-1.ult2.sydney.telstra.net (203.50.20.51) 30.927
> ms 31.323 ms 31.311 ms
>
> 4 vocusf.lnk.telstra.net (139.130.73.106) 33.533 ms 33.951 ms 34.054
> ms
>
> 5 ten-0-0-0-1.cor03.syd03.nsw.VOCUS.net.au (175.45.72.72) 34.961 ms
> 36.609 ms 36.945 ms
>
> 6 ten-0-1-0.bdr02.syd03.nsw.VOCUS.net.au (114.31.192.61) 38.232 ms
> 18.997 ms 19.417 ms
>
> 7 as45671.cust.bdr02.syd03.nsw.VOCUS.net.au (114.31.200.126) 21.353 ms
> 21.467 ms 25.016 ms
>
> 8 bdr-1.nsw.au.glonetwork.com.au (27.50.72.99) 25.261 ms 25.948 ms
> 26.251 ms
>
> 9 103.18.207.24 (103.18.207.24) 29.817 ms 32.019 ms 32.166 ms
>
>
>
> Also fine on SAU ;)
>
>
>
> damien at echelon:~$ traceroute 103.18.207.24
>
> traceroute to 103.18.207.24 (103.18.207.24), 30 hops max, 60 byte packets
>
> 1 bdr01-vlan49.syd01.nsw.rendrag.net.au (103.235.52.49) 0.501 ms
> 0.472 ms 0.305 ms
>
> 2 core-bdr-1-1-ve309.sau.net.au (27.50.65.210) 0.654 ms 0.583 ms
> 0.594 ms
>
> 3 202.130.32.77 (202.130.32.77) 0.605 ms 0.688 ms 0.792 ms
>
> 4 bdr-1.nsw.au.glonetwork.com.au (27.50.72.99) 9.143 ms 9.133 ms
> 9.113 ms
>
> 5 103.18.207.24 (103.18.207.24) 11.992 ms 12.484 ms 12.470 ms
>
>
>
> Route via Eftel is a little hinkey, but works..
>
>
>
> # ADDRESS LOSS SENT LAST AVG BEST
> WORST STD-DEV STATUS 1
> vlan151.bdr01.syd01.eftel.com 0% 4 0.9ms 1 0.9
> 1.2 0.1 2
> ge0-1-902.lns01.per01.eftel.com 0% 4 59.6ms 59.5 59.3
> 59.6 0.1 3
> vocus.ix.waia.asn.au 0% 4 61ms 59.9 59.3
> 61 0.7 4
> ge-0-0-0-18.cor01.per02.wa.vo... 0% 4 56ms 56.1 55.9 56.6
> 0.3 5
> ten-0-2-0-7.cor03.syd03.nsw.v... 0% 4 56.7ms 56.5 56.2 56.7
> 0.2 6
> ten-0-1-0.bdr02.syd03.nsw.voc... 0% 4 55.7ms 55.6 55.6 55.7
> 0 7
> as45671.cust.bdr02.syd03.nsw.... 0% 3 55.8ms 56.1 55.7 56.8
> 0.5 8
> bdr-1.nsw.au.glonetwork.com.au 66.. 3 timeout 57.4 57.4
> 57.4 0 9
> 103.18.207.24 0% 3 60.4ms 61.1 59.4 63.6
> 1.8
>
>
>
> On 23 August 2014 23:01, Daniel Watson <Daniel at glovine.com.au
> <javascript:_e(%7B%7D,'cvml','Daniel at glovine.com.au');>> wrote:
>
> Just curious if anybody else is seeing any other issues with other
> providers? TPG, AAPT or EQX?
>
>
>
>
>
> C:\Users\danie_000>tracert 203.26.189.240
>
>
>
> Tracing route to 203.26.189.240 over a maximum of 30 hops
>
>
>
> 1 14 ms 16 ms 4 ms 192.168.1.1
>
> 2 29 ms 27 ms 35 ms 10.20.21.170
>
> 3 30 ms 40 ms 81 ms nme-sot-dry-crt1-Te0-3-0-14.tpgi.com.au
> [203.26.22.225]
>
> 4 28 ms 31 ms 27 ms 203.8.183.21
>
> 5 38 ms 38 ms 40 ms po9.mflindist02.aapt.net.au [202.10.14.32]
>
> 6 37 ms 42 ms 39 ms te2-1-110.mflindist01.aapt.net.au
> [202.10.12.136]
>
> 7 * 37 ms 37 ms te0-3-4-0.mflincore01.aapt.net.au
> [202.10.12.11]
>
> 8 48 ms 38 ms 37 ms bu2.sglebcore01.aapt.net.au [202.10.10.72]
>
> 9 42 ms 43 ms 41 ms po11.sglebdist01.aapt.net.au [202.10.12.6]
>
> 10 46 ms 39 ms 36 ms 1-1-1.sglebinte01.aapt.net.au
> [203.131.60.39]
>
> 11 39 ms 45 ms 42 ms 203-174-189-250.syd.static-ipl.aapt.com.au
> [203.
>
> 174.189.250]
>
> 12 41 ms 41 ms 38 ms xe-0-1-1.gw102.sy1.ap.equinix.com
> [27.111.240.146]
>
> 13 49 ms 40 ms 44 ms bdr-1-1-te2-49.sau.net.au [203.190.232.234]
>
> 14 37 ms 41 ms 37 ms 202.130.32.77
>
> 15 40 ms 49 ms 44 ms bdr-1.nsw.au.glonetwork.com.au
> [27.50.72.99]
>
> 16 41 ms 43 ms 40 ms
> bdr-1.nsw.au.dedicatedhosting-solutions.com.au [
>
> 103.18.205.241]
>
> 17 53 ms 46 ms 46 ms 203.26.189.240
>
>
>
> Trace complete.
>
>
>
> Above works,
>
>
>
> C:\Users\danie_000>tracert 103.18.207.24
>
>
>
> Tracing route to 103.18.207.24 over a maximum of 30 hops
>
>
>
> 1 5 ms 4 ms 4 ms 192.168.1.1
>
> 2 28 ms 34 ms 28 ms 10.20.21.170
>
> 3 27 ms 26 ms 37 ms nme-sot-dry-agr2-Te2-6.tpgi.com.au
> [202.7.173.69]
>
> 4 32 ms 35 ms 30 ms nme-sot-dry-crt2-Te0-1-0-4.tpgi.com.au
> [203.26.22.49]
>
> 5 45 ms 42 ms 38 ms syd-sot-ken-crt1-te-gi-0-6-0-2.tpgi.com.au
> [203.219.106.189]
>
> 6 40 ms 38 ms 38 ms 203-219-35-4.static.tpgi.com.au
> [203.219.35.4]
>
> 7 * * * Request timed out.
>
> 8 * * * Request timed out.
>
> 9 * * * Request timed out.
>
> 10 * * * Request timed out.
>
> 11 * * * Request timed out.
>
> 12 * * * Request timed out.
>
> 13 * * * Request timed out.
>
> 14 * * * Request timed out.
>
> 15 * * * Request timed out.
>
> 16 * * * Request timed out.
>
> 17 * * * Request timed out.
>
> 18 * * * Request timed out.
>
> 19 * * * Request timed out.
>
> 20 * * * Request timed out.
>
> 21 * * * Request timed out.
>
> 22 * * * Request timed out.
>
> 23 * * * Request timed out.
>
> 24 * * * Request timed out.
>
> 25 * * * Request timed out.
>
> 26 te0-1-0-3.ccr22.lax04.atlas.cogentco.com [38.104.210.53] reports:
> Destinat
>
> ion net unreachable.
>
>
>
> Trace complete.
>
>
>
> Above doesnt
>
>
>
>
>
> Regards,
>
> Daniel Watson
>
> Network Administrator / Network Operations Manager
>
>
>
> E Daniel at GloVine.com.au
> <javascript:_e(%7B%7D,'cvml','Daniel at GloVine.com.au');>
>
> W www.GloVine.com.au
>
>
>
> *From:* Andrew Cox [mailto:andrew.cox at bigair.net.au
> <javascript:_e(%7B%7D,'cvml','andrew.cox at bigair.net.au');>]
> *Sent:* Saturday, 23 August 2014 10:56 PM
> *To:* Jared Hirst
> *Cc:* Daniel Watson; Ausnog at ausnog.net
> <javascript:_e(%7B%7D,'cvml','Ausnog at ausnog.net');>
>
>
> *Subject:* Re: [AusNOG] Vocus international service outage
>
>
>
> Likewise we've dropped Vocus entirely for now until we hear more.
>
> It looked like although we weren't receiving any traffic, that they were
> still advertising our prefixes upstream.
>
> - Andrew
>
>
>
> On 23 August 2014 22:41, Jared Hirst <jared.hirst at serversaustralia.com.au
> <javascript:_e(%7B%7D,'cvml','jared.hirst at serversaustralia.com.au');>>
> wrote:
>
> We have just turned Vocus off. Using other providers for now, as the
> flapping is causing it to go up and down.
>
>
>
> On Sat, Aug 23, 2014 at 10:37 PM, Daniel Watson <Daniel at glovine.com.au
> <javascript:_e(%7B%7D,'cvml','Daniel at glovine.com.au');>> wrote:
>
> Indeed seeing some big drops in gaming traffic at present, normally we
> see above 60mbit on weekends at evenings, but not even seeing 40mbit at
> present :S
>
>
>
>
>
> Regards,
>
> Daniel Watson
>
> Network Administrator / Network Operations Manager
>
>
>
> E Daniel at GloVine.com.au
> <javascript:_e(%7B%7D,'cvml','Daniel at GloVine.com.au');>
>
> W www.GloVine.com.au
>
>
>
> *From:* AusNOG [mailto:ausnog-bounces at lists.ausnog.net
> <javascript:_e(%7B%7D,'cvml','ausnog-bounces at lists.ausnog.net');>] *On
> Behalf Of *Jared Hirst
> *Sent:* Saturday, 23 August 2014 10:35 PM
>
>
> *To:* Andrew Cox
> *Cc:* Ausnog at ausnog.net
> <javascript:_e(%7B%7D,'cvml','Ausnog at ausnog.net');>
> *Subject:* Re: [AusNOG] Vocus international service outage
>
>
>
> Yeah we are seeing this! Everything running via them is flapping, they
> claim to have 'routed around it' but that's not the case. Very frustrating
> as it's been an hour and no one there seems to know whats going on....
>
>
>
> On Sat, Aug 23, 2014 at 10:29 PM, Andrew Cox <andrew.cox at bigair.net.au
> <javascript:_e(%7B%7D,'cvml','andrew.cox at bigair.net.au');>> wrote:
>
> Hey All,
>
> Just saw the dashboards light up with connectivity issues internationally
> for Vocus services and thought I'd make others aware.
>
> Vocus outage report is saying: "core network link between 59 Doody Street,
> Alexandria and 55 South Market Street, San Jose has failed" which hopefully
> isn't a Southern Cross fault!
>
> Anyone else seeing this or have more info?
>
>
>
> Cheers,
>
> Andrew
>
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> <javascript:_e(%7B%7D,'cvml','AusNOG at lists.ausnog.net');>
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
>
>
>
> --
>
>
>
>
>
>
>
>
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> <javascript:_e(%7B%7D,'cvml','AusNOG at lists.ausnog.net');>
> http://lists.ausnog.net/mailman/listinfo/ausnog
>
>
>
>
>
> --
>
> Damien Gardner Jnr
> VK2TDG. Dip EE. GradIEAust
> rendrag at rendrag.net <javascript:_e(%7B%7D,'cvml','rendrag at rendrag.net');>
> - http://www.rendrag.net/
> --
> We rode on the winds of the rising storm,
> We ran to the sounds of thunder.
> We danced among the lightning bolts,
> and tore the world asunder
>
--
*Trent Farrell*
*Riot Games*
*IP Network Engineer*
E: tfarrell at riotgames.com | IE: +353 83 446 6809 | US: +1 424 285 9825
Summoner name: Foro
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20140824/82d4ff20/attachment.html>
More information about the AusNOG
mailing list