<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<style type="text/css" style="display:none;"> #aqm-original P {
margin-top:0;margin-bottom:0;
} /* style */
</style>
</head>
<body>
<div dir="auto">
<div dir="auto">Reminds me of Vocus Sydney way back when. Doesn't sound brilliantly engineered - very curious for the explanation down the track to see if it could've been avoided. </div><div dir="auto"><br></div><div dir="auto">Re: single feed, transfer switches have their place for single corded kit, have no doubt! </div><div dir="auto"><br></div><div dir="auto">Cheers, </div><div dir="auto"><br></div><div id="aqm-signature" dir="auto" style="color: black;"><div dir="auto">Luke Thompson</div><div dir="auto">Chief Technical Officer</div><div dir="auto"><br></div><div dir="auto">The Network Crew Pty Ltd</div><div dir="auto"><a href="https://thenetworkcrew.com.au">https://thenetworkcrew.com.au</a></div></div><div dir="auto"><br></div>
<div id="aqm-original" style="color: black;">
<!-- body start -->
<div class="aqm-original-body">
<div style="color: black;">
<p style="color: black; font-size: 10pt; font-family: sans-serif; margin: 8pt 0;">On 13 March 2023 10:39:58 am James Braunegg <james.braunegg@micron21.com> wrote:</p>
<blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p></p>
<p class="MsoNormal">Dear Luke<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">This is more than likely related to a power issue at 530 Collins Street we lost at 8:45am this morning supply A and then around 8:55am we also lost supply B for all services running at this location.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Power retuned around 9:45am for supply B however supply A was still dead… then at 10:20am both supply A and B both went down again….<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Currently it looks like supply B is back online as of 10:33am… but supply A is still offline so anyone using single power equipment on supply A only will be having issues !<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I have raised a ticket with the vendor however no reply as of yet…. Will have to wait for the PIR to hear what happened.</p>
<p class="MsoNormal"><br>
</p>
<p class="MsoNormal"><span style="font-size: 12pt;"> </span><span style="font-size: 12pt;">Kindest Regards</span></p>
<p class="MsoNormal"><o:p></o:p></p>
<br>
<p></p>
<p></p>
<p class="MsoNormal"><o:p><br>
</o:p></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse;mso-yfti-tbllook:1184;mso-padding-alt:0cm 0cm 0cm 0cm">
<tbody>
<tr>
<td style="padding:0cm 0cm 11.25pt 0cm">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse;mso-yfti-tbllook:1184;mso-padding-alt:0cm 0cm 0cm 0cm">
<tbody>
<tr>
<td valign="top" style="padding:0cm 0cm 0cm 0cm">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse;mso-yfti-tbllook:1184;mso-padding-alt:
0cm 0cm 0cm 0cm">
<tbody>
<tr>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:15.0pt"><b><span style="font-size:
11.5pt;font-family:"Arial",sans-serif;color:#64717A;mso-fareast-language:
EN-US">James Braunegg<o:p></o:p></span></b></p>
</td>
</tr>
<tr>
<td style="padding:0cm 0cm 0cm 0cm"></td>
</tr>
</tbody>
</table>
</td>
<td valign="top" style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:105%"><img width="17" height="38" v:shapes="Picture_x0020_1" style="user-select: none;"><span style="font-size: 12pt; line-height: 105%;"><o:p></o:p></span></p>
</td>
<td valign="top" style="padding:0cm 0cm 0cm 0cm">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse;mso-yfti-tbllook:1184;mso-padding-alt:
0cm 0cm 0cm 0cm">
<tbody>
<tr>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:12.75pt"><a href="tel:1300%20769%20972" title="tel:1300%20769%20972
Ctrl+Click or tap to follow the link"><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#64717A">1300 769 972</span></a><span style="font-size: 9pt; font-family: Arial, sans-serif;"> /
</span><a href="tel:1300%20769%20972" title="tel:1300%20769%20972
Ctrl+Click or tap to follow the link"><span style="font-size:9.0pt;font-family:
"Arial",sans-serif;color:#64717A">0488 997 207</span></a><span style="font-size: 9pt; font-family: Arial, sans-serif;"><o:p></o:p></span></p>
</td>
</tr>
<tr>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:12.75pt"><u><span style="font-size:
9.0pt;font-family:"Arial",sans-serif;color:#0563C1;mso-fareast-language:
EN-US"><a href="mailto:james.braunegg@micron21.com"><span style="color:#0563C1">james.braunegg@micron21.com</span></a></span></u><span style="font-size: 9pt; font-family: Arial, sans-serif;"><o:p></o:p></span></p>
</td>
</tr>
<tr>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:12.75pt"><span style="font-size: 9pt; font-family: Arial, sans-serif;"><a href="https://www.ddosprotection.com.au/"><span style="color:#0563C1">www.ddosprotection.com.au
</span></a> <o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
</td>
<td valign="top" style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:105%"><a href="http://www.micron21.com/"><span style="text-decoration-line: none;"></span></a><span style="font-size: 12pt; line-height: 105%;"><o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td style="padding:0cm 0cm 18.75pt 0cm">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse;mso-yfti-tbllook:1184;mso-padding-alt:0cm 0cm 0cm 0cm">
<tbody>
<tr>
<td width="18" valign="top" style="width:13.5pt;padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:105%"><a href="https://www.facebook.com/micron21/" title=""Follow Micron21 on Facebook" "><span style="text-decoration-line: none;"><img border="0" width="13" height="17" v:shapes="Picture_x0020_3" style="user-select: none;"></span></a><span style="font-size: 12pt; line-height: 105%;"><o:p></o:p></span></p>
</td>
<td width="30" valign="top" style="width:22.5pt;padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:105%"><a href="https://twitter.com/micron21" title=""Follow Micron21 on Twitter" "><span style="text-decoration-line: none;"><img border="0" width="17" height="17" v:shapes="Picture_x0020_4" style="user-select: none;"></span></a><o:p></o:p></p>
</td>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:12.75pt"><span style="font-size:9.0pt;
font-family:"Arial",sans-serif;color:#64717A;mso-fareast-language:EN-US">Follow us on <a href="https://m21status.com"><span style="color:#0563C1">m21status.com</span></a> for
important service and system updates.<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td style="padding:0cm 0cm 0cm 0cm">
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="490" style="width:367.5pt;border-collapse:collapse;mso-yfti-tbllook:1184;
mso-padding-alt:0cm 0cm 0cm 0cm">
<tbody>
<tr>
<td style="padding:0cm 0cm 0cm 0cm">
<p class="MsoNormal" style="line-height:12.75pt"><span style="font-size:9.0pt;
font-family:"Arial",sans-serif;color:#64717A;mso-fareast-language:EN-US">This message is intended for the addressee named above. It may contain privileged or confidential information.
If you are not the intended recipient of this message you must not use, copy, distribute or disclose it to anyone other than the addressee. If you have received this message in error please return the message to the sender by replying to it and then delete
the message from your computer.<o:p></o:p></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<br>
<p></p>
</div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><span style="font-family: Calibri, sans-serif; color: #000000;"><b>From:</b> AusNOG <ausnog-bounces@lists.ausnog.net> on behalf of Luke Thompson <luke.t@tncrew.com.au><br>
<b>Sent:</b> Monday, 13 March 2023 10:37:03 AM<br>
<b>To:</b> Damien Gardner Jnr; Chris Wright<br>
<b>Cc:</b> ausnog@ausnog.net<br>
<b>Subject:</b> Re: [AusNOG] Aussie Broadband Outage</span>
<div> </div>
</div>
<div>
<div dir="auto">
<div dir="auto">Given the commonality of outage type, it's smelling like civil or grid problems<span style="font-size: 12pt;"> isn't it?</span></div>
<div dir="auto"><span style="font-size: 12pt;"><br>
</span></div>
<div dir="auto"><span style="font-size: 12pt;">Vocus and Equinix losing power over both rails, optical links dropping, etc sounds like a major problem in the earth or at a dist/sub station. Curious to learn the root cause. </span></div>
<div dir="auto"><span style="font-size: 12pt;"><br>
</span></div>
<div dir="auto"><span style="font-size: 12pt;">A team somewhere is having a bad Monday morning, that's for sure. Good luck to all. </span></div>
<div dir="auto"><br>
</div>
<div dir="auto" style="color: black;">
<div dir="auto">Luke Thompson</div>
<div dir="auto">Chief Technical Officer</div>
<div dir="auto"><br>
</div>
<div dir="auto">The Network Crew Pty Ltd</div>
<div dir="auto"><a href="https://thenetworkcrew.com.au">https://thenetworkcrew.com.au</a></div>
</div>
<div dir="auto"><br>
</div>
<div dir="auto"><br>
</div>
<div style="color: black;">
<div class="aqm-original-body">
<div style="color: black;">
<p style="color: black; font-size: 10pt; font-family: sans-serif; margin: 8pt 0;">
On 13 March 2023 10:31:35 am Damien Gardner Jnr <rendrag@rendrag.net> wrote:</p>
<blockquote type="cite" class="gmail_quote" style="margin: 0 0 0 0.75ex; border-left: 1px solid #808080; padding-left: 0.75ex;">
<div dir="auto">I would assume it’s to do with VDC-MEL03 having lost power around 9:05am. A few folks reported getting one rail back temporarily, and then lost power again. We’ve not seen power restore at all so far.</div>
<div dir="auto"><br>
</div>
<div dir="auto">—DG</div>
<div><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, 13 Mar 2023 at 10:28 am, Chris Wright <<a href="mailto:chris@securityshift.com">chris@securityshift.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)">
I also had outages in Melbourne, (Equinix ME2) around the same time (~9:00 - > ~9:45am).<br>
<br>
Still got a link down with Vocus and still having links go up and down with Swoop.<br>
<br>
Something more significant is happening somewhere - would be great if anyone can shed some light on what's happening so we all don't keep chasing our tails.<br>
<br>
Thanks<br>
<br>
c.<br>
<br>
-----Original Message-----<br>
From: AusNOG <<a href="mailto:ausnog-bounces@lists.ausnog.net" target="_blank">ausnog-bounces@lists.ausnog.net</a>> On Behalf Of DaZZa<br>
Sent: Monday, 13 March 2023 10:21<br>
To: Luke Thompson <<a href="mailto:luke.t@tncrew.com.au" target="_blank">luke.t@tncrew.com.au</a>><br>
Cc: <a href="mailto:ausnog@ausnog.net" target="_blank">ausnog@ausnog.net</a> <<a href="mailto:ausnog@lists.ausnog.net" target="_blank">ausnog@lists.ausnog.net</a>><br>
Subject: Re: [AusNOG] Aussie Broadband Outage<br>
<br>
[You don't often get email from <a href="mailto:dazzagibbs@gmail.com" target="_blank">
dazzagibbs@gmail.com</a>. Learn why this is important at <a href="https://aka.ms/LearnAboutSenderIdentification" rel="noreferrer" target="_blank">
https://aka.ms/LearnAboutSenderIdentification</a> ]<br>
<br>
This email came from an external source. Please be cautious with links and content.<br>
<br>
Interesting.<br>
<br>
I believe Aussie is based in Melbourne - I had a major outage on several Vocus services in Melbourne between about 09:10 and 09:40 this morning.<br>
<br>
Wonder if they're in a shared DC and took a hit at the same time as Aussie did...<br>
<br>
D<br>
<br>
On Mon, 13 Mar 2023 at 10:10, Luke Thompson <<a href="mailto:luke.t@tncrew.com.au" target="_blank">luke.t@tncrew.com.au</a>> wrote:<br>
><br>
> James on Whirlpool has just shared what you likely saw tweeted before:<br>
><br>
> "We are currently experiencing issues with one of our data centres, which is impacting some of our services including MyAussie, VoIP, and our Customer Service phone lines / Live Chat"<br>
><br>
> Sounds like there's no redundancy for Carbon, My Aussie, VoIP, Chat, etc. Wild.<br>
><br>
> Luke Thompson<br>
> Chief Technical Officer<br>
><br>
> The Network Crew Pty Ltd<br>
> <a href="https://thenetworkcrew.com.au" rel="noreferrer" target="_blank">https://thenetworkcrew.com.au</a><br>
><br>
> On 13 March 2023 10:07:57 am Troy <<a href="mailto:troy@troykelly.com" target="_blank">troy@troykelly.com</a>> wrote:<br>
>><br>
>> Is anybody from Aussie able to share what's going on?<br>
>><br>
>> Looks like there's a major outage with services (no Carbon, phones, <br>
>> etc)<br>
>><br>
>> There was? a tweet for about 40 seconds saying that there was an issue with a data centre, but it was deleted.<br>
>><br>
>> Regards, Troy<br>
>> Brevity is the elixir of life.<br>
>> Father Hector McGrath, Pixie 2020<br>
><br>
><br>
> _______________________________________________<br>
> AusNOG mailing list<br>
> <a href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a><br>
> <a href="https://lists.ausnog.net/mailman/listinfo/ausnog" rel="noreferrer" target="_blank">
https://lists.ausnog.net/mailman/listinfo/ausnog</a><br>
<br>
<br>
<br>
--<br>
veg*e*tar*i*an:<br>
Ancient tribal slang for the village idiot who can't hunt, fish or ride _______________________________________________<br>
AusNOG mailing list<br>
<a href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a><br>
<a href="https://lists.ausnog.net/mailman/listinfo/ausnog" rel="noreferrer" target="_blank">https://lists.ausnog.net/mailman/listinfo/ausnog</a><br>
_______________________________________________<br>
AusNOG mailing list<br>
<a href="mailto:AusNOG@lists.ausnog.net" target="_blank">AusNOG@lists.ausnog.net</a><br>
<a href="https://lists.ausnog.net/mailman/listinfo/ausnog" rel="noreferrer" target="_blank">https://lists.ausnog.net/mailman/listinfo/ausnog</a><br>
</blockquote>
</div>
</div>
-- <br>
<div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<p>Damien Gardner Jnr<br>
VK2TDG. Dip EE. GradIEAust<br>
<a href="mailto:rendrag@rendrag.net" target="_blank">rendrag@rendrag.net</a> - <span><a href="http://www.rendrag.net/" target="_blank">http://www.rendrag.net/</a><u><br>
</u></span>--<br>
We rode on the winds of the rising storm,<br>
We ran to the sounds of thunder.<br>
We danced among the lightning bolts,<br>
and tore the world asunder</p>
</div>
</div>
</blockquote>
</div>
</div>
</div>
<div dir="auto"><br>
</div>
</div>
</div>
<div>_______________________________________________</div>
<div>AusNOG mailing list</div>
<div><a class="aqm-autolink aqm-autowrap" href="mailto:AusNOG%40lists.ausnog.net">AusNOG@lists.ausnog.net</a></div>
<div><a class="aqm-autolink aqm-autowrap" href="https://lists.ausnog.net/mailman/listinfo/ausnog">https://lists.ausnog.net/mailman/listinfo/ausnog</a></div>
<div><br></div></blockquote>
</div>
</div>
<!-- body end -->
</div><div dir="auto"><br></div>
</div></body>
</html>