[AusNOG] Vocus Datacentre outage.

Tim Dykes ttdykes at gmail.com
Sat Mar 18 09:33:18 AEDT 2023 

Current outage is unrelated, suspected fibre cut in Melbourne... Incase
others are checking this thread for the same reason I just did.

On Thu, 16 Mar 2023, 1:14 pm DaZZa, <dazzagibbs at gmail.com> wrote:

> Nope. I've asked for an RFO/PIR for the outage, and been put in the queue
> for when they actually produce it.
>
> D
>
> On Thu, 16 Mar 2023 at 12:51, Andrew Welch <mail at andrewwelch.net> wrote:
>
>> Has anyone heard any details about the planned UPS maintenance that's
>> scheduled for tomorrow at VDC-MEL03? Only info I've seen is what's in the
>> last outage update, which is just that UPS-A is being replaced, nothing on
>> whether this is service impacting or anything like that.
>>
>> Thanks,
>> Andrew Welch
>> mail at andrewwelch.net
>>
>>
>> On Tue, 14 Mar 2023 at 13:15, Jared Hirst | Servers Australia <
>> jared.hirst at serversaustralia.com.au> wrote:
>>
>>> Good times back then Joel! It was a UPS issue back there too if I
>>> remember, and the A and B feeds were all fed into the one UPS….
>>>
>>> A tonne of people shifted within that week, for me it was the way that
>>> it was palmed off, as you say, others might have issues but they own those
>>> issues and rectify them…
>>>
>>>
>>>
>>>
>>>
>>> Jared Hirst
>>> Chief Executive Officer
>>> P: (02) 8115 8801
>>> 4 Amy Close, Wyong, NSW 2259
>>> Need assistance? We are here 24/7 +61 2 8115 8888
>>> [image: Sent from Front]
>>>
>>> On March 14, 2023 at 12:26 PM GMT+11 joel at zonenetworks.com.au wrote:
>>>
>>> In EQX defence Sy1 didn’t go dark… it was multiple (3-5) rows of rack
>>> that lost power in 2009.
>>>
>>>
>>>
>>> We were one of the customers heavily impacted in 2009 but understood the
>>> reason why it happened and EQX fixed it all up.
>>>
>>>
>>>
>>> Vocus Doody st different story and now melb
>>>
>>>
>>>
>>> We pulled out of Doody after the drama we faced there.
>>>
>>>
>>>
>>> Joel.
>>>
>>>
>>>
>>> *From:* AusNOG <ausnog-bounces at lists.ausnog.net> *On Behalf Of *Nathan
>>> Brookfield
>>> *Sent:* Tuesday, 14 March 2023 11:54 AM
>>> *To:* Christopher Hawker <chris at thesysadmin.dev>
>>> *Cc:* ausnog at lists.ausnog.net
>>> *Subject:* Re: [AusNOG] Vocus Datacentre outage.
>>>
>>>
>>>
>>> I knew I didn’t remember it for a reason 😂
>>>
>>>
>>>
>>> *[AusNOG] Equinix Outage
>>> <https://lists.ausnog.net/pipermail/ausnog/2009-July/003239.html>*
>>>
>>> *lists.ausnog.net
>>> <https://lists.ausnog.net/pipermail/ausnog/2009-July/003239.html>*
>>>
>>> <https://lists.ausnog.net/pipermail/ausnog/2009-July/003239.html>
>>>
>>>
>>>
>>> On 14 Mar 2023, at 10:50, Christopher Hawker <chris at thesysadmin.dev>
>>> wrote:
>>>
>>>
>>>
>>> 
>>>
>>> SY1 apparently experienced a full power outage back in July 2009,
>>> according to AusNOG Archives. Some reported full outages, some partial. Was
>>> not a DC fault though, looks like a much larger grid issue where a fault at
>>> the Bayswater Power Station in the Hunter Valley (which affected generators
>>> in QLD and VIC) caused about 1000MW to be dropped from the grid, resulting
>>> in rolling blackouts across QLD, NSW, TAS, SA and VIC.
>>>
>>>
>>>
>>> https://lists.ausnog.net/pipermail/ausnog/2009-July/003222.html
>>>
>>>
>>>
>>> CH.
>>> ------------------------------
>>>
>>> *From:* AusNOG <ausnog-bounces at lists.ausnog.net> on behalf of Nathan
>>> Brookfield <Nathan.Brookfield at iperium.com.au>
>>> *Sent:* Tuesday, March 14, 2023 11:40 AM
>>> *To:* Matt Perkins <matt at spectrum.com.au>; ausnog at lists.ausnog.net <
>>> ausnog at lists.ausnog.net>
>>> *Subject:* Re: [AusNOG] Vocus Datacentre outage.
>>>
>>>
>>>
>>> Have 100% had A and B fail separately at SY1 but never both,
>>> thankfully!!!
>>>
>>>
>>>
>>> Could have been part of the facility though so don’t quote me.
>>>
>>>
>>>
>>> UK on the other hand, fail!
>>>
>>>
>>>
>>> *From:* Matt Perkins <matt at spectrum.com.au>
>>> *Sent:* Tuesday, 14 March 2023 11:30 AM
>>> *To:* Nathan Brookfield <Nathan.Brookfield at iperium.com.au>;
>>> ausnog at lists.ausnog.net
>>> *Subject:* Re: [AusNOG] Vocus Datacentre outage.
>>>
>>>
>>>
>>> SY1 had A or B fail a few years back if my memory serves me correctly.
>>> But I was referring  London and Manchester which both had total outages in
>>> recent years.
>>>
>>> Matt.
>>>
>>>
>>>
>>> On 14/3/2023 11:26 am, Nathan Brookfield wrote:
>>>
>>> I’ve never had an A+B power rail failure at Equinix in 18 years I’ve
>>> been in there facilities, TOUCH WOOD!!!
>>>
>>>
>>>
>>>
>>> *Nathan Brookfield | VK2NAB *General Manager
>>>
>>> *p*: 1300 592 330 | *m*: 0412 266 008
>>> *e*: Nathan.brookfield at iperium.com.au | *w*: iperium.com.au
>>>
>>> Suite 4.02, 189 Kent Street Sydney NSW 2000
>>>
>>> <image001.png>
>>>
>>>
>>> *Your Telco Team*
>>>
>>> DISCLAIMER: This document is intended solely for the named addressee.
>>> This electronic communication, which includes any files or attachments
>>> thereto, contains proprietary or confidential information and may be
>>> privileged and otherwise protected under copyright or other applicable
>>> intellectual property laws. The use, disclosure, copying or distribution of
>>> any of the information contained in this document, by any person other than
>>> the addressee, is strictly prohibited. If you received this document in
>>> error, please contact the sender immediately and delete all the material
>>> from any computer. Confidentiality and legal privilege are not waived or
>>> lost by reason of mistaken delivery to you. Any views or opinions presented
>>> are solely those of the author and do not necessarily represent those of
>>> Iperium.
>>>
>>> WARNING: Computer viruses can be transmitted via email. The recipient
>>> should check this email and any attachments for the presence of viruses.
>>> Iperium accepts no liability for any damage caused by any virus transmitted
>>> by this email.
>>>
>>> *From:* AusNOG <ausnog-bounces at lists.ausnog.net>
>>> <ausnog-bounces at lists.ausnog.net> *On Behalf Of *Matt Perkins
>>> *Sent:* Tuesday, 14 March 2023 11:18 AM
>>> *To:* ausnog at lists.ausnog.net
>>> *Subject:* Re: [AusNOG] Vocus Datacentre outage.
>>>
>>>
>>>
>>> Yea because Eqinix has never had a power outage. :)
>>>
>>> I think it's a little premature to judge at the moment. Doesn't sound
>>> good based on the current info but we have all had bad days im sure.
>>>
>>> Matt
>>>
>>>
>>>
>>> On 14/3/2023 11:06 am, Bradley Amm wrote:
>>>
>>> Friends like let friends put gear in a Vocus dtacenter. Only NextDC or
>>> Equinix :)
>>>
>>> Sent from my iPhone
>>>
>>>
>>>
>>> On 14 Mar 2023, at 7:08 am, DaZZa <dazzagibbs at gmail.com>
>>> <dazzagibbs at gmail.com> wrote:
>>>
>>> 
>>>
>>> Doesn't say a whole lot about Vocus' data center design methodology, nor
>>> their periodic contingency testing.
>>>
>>>
>>>
>>> They're apparently replacing the entire UPS on Friday - guess we can all
>>> expect more outages then when they stuff that up.
>>>
>>>
>>>
>>> D
>>>
>>>
>>>
>>> On Mon, 13 Mar 2023 at 18:30, Chad Kelly <chad at cpkws.com.au> wrote:
>>>
>>> Thought I would create a new thread on this, as the issue wasn’t related
>>> to AussieBB.
>>>
>>> It is looking like this Melbourne outage is similar to the Sydney outage
>>> back in 2018, in that it appears to have been yet again caused by an issue
>>> with their UPS systems yet again.
>>>
>>> I’ll leave this hear
>>> https://www.arnnet.com.au/article/633314/servers-australia-uproots-services-after-vocus-data-centre-outage/
>>> and this article on today’s outage as well.
>>>
>>>
>>> https://www.itnews.com.au/news/vocus-melbourne-data-centre-hit-by-power-outage-591987
>>>
>>>
>>>
>>> Regards Chad.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Chad Kelly
>>>
>>> Manager
>>>
>>> CPK Web Services
>>>
>>> Phone 03 52730246
>>>
>>> Web https://www.cpkws.com.au
>>>
>>> <image001.jpg>
>>>
>>>
>>>
>>> _______________________________________________
>>> AusNOG mailing list
>>> AusNOG at lists.ausnog.net
>>> https://lists.ausnog.net/mailman/listinfo/ausnog
>>>
>>>
>>>
>>> --
>>>
>>> veg·e·tar·i·an:
>>> Ancient tribal slang for the village idiot who can't hunt, fish or ride
>>>
>>> _______________________________________________
>>> AusNOG mailing list
>>> AusNOG at lists.ausnog.net
>>> https://lists.ausnog.net/mailman/listinfo/ausnog
>>>
>>>
>>>
>>> _______________________________________________
>>>
>>> AusNOG mailing list
>>>
>>> AusNOG at lists.ausnog.net
>>>
>>> https://lists.ausnog.net/mailman/listinfo/ausnog
>>>
>>> --
>>>
>>> ** DO NOT SEND PRIVATE OR CONFIDENTIAL MATERIAL VIA EMAIL
>>>
>>>
>>>
>>> /* Matt Perkins
>>>
>>>         Direct 02 8916 8101     Spectrum Networks Ptd. Ltd.
>>>
>>>         Office 1300 133 299     matt at spectrum.com.au
>>>
>>>         ABN 66 090 112 913      Level 6, 350 George Street Sydney 2000
>>>
>>>
>>>
>>> */
>>>
>>> --
>>>
>>> ** DO NOT SEND PRIVATE OR CONFIDENTIAL MATERIAL VIA EMAIL
>>>
>>>
>>>
>>> /* Matt Perkins
>>>
>>>         Direct 02 8916 8101     Spectrum Networks Ptd. Ltd.
>>>
>>>         Office 1300 133 299     matt at spectrum.com.au
>>>
>>>         ABN 66 090 112 913      Level 6, 350 George Street Sydney 2000
>>>
>>>
>>>
>>> */
>>>
>>>
>>> _______________________________________________
>>> AusNOG mailing list
>>> AusNOG at lists.ausnog.net
>>> https://lists.ausnog.net/mailman/listinfo/ausnog
>>>
>> _______________________________________________
>> AusNOG mailing list
>> AusNOG at lists.ausnog.net
>> https://lists.ausnog.net/mailman/listinfo/ausnog
>>
>
>
> --
> veg·e·tar·i·an:
> Ancient tribal slang for the village idiot who can't hunt, fish or ride
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> https://lists.ausnog.net/mailman/listinfo/ausnog
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ausnog.net/pipermail/ausnog/attachments/20230318/d64a0021/attachment.htm>

More information about the AusNOG mailing list