[AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August

Shaun McGuane shaun at rackcentral.com
Thu Aug 10 10:07:03 EST 2017 

HI Simon,

Thank you for your input to the list.

While you may have provided notifications well in advance, your schedule provided has been changed multiple times and this creates confusion.
Without asking for further clarification last night I would never have received the following updated schedule from Craig Armstrong today.

Data Hall 2 – B feed (west/blue PDU side) Tuesday 15th Aug, A feed has been pushed back to later date around November, you will receive official notice once new date confirmed.
Data Hall 3 –A and B feed has been pushed back to later date around October/November, you will receive official notice once new date confirmed.
Data Hall 4 –A and B feed has been pushed back to later date around October/November, you will receive official notice once new date confirmed.

From the latest update today, It appears that most of the works are being postponed for the time being

Not all devices within racks support 2 feeds and planning is involved with these devices, which is why it is crucial to receive accurate information about any pending outages or upgrades
to the NextDC DataCentres which affect services.

Additionally, I have not received these updates via the normal NextDC E-Mail communication method, so I am sure other customers will be wondering whats happening too.

Happy to have a conversation offline about this feel free to give me a call on my mobile.

Regards
Shaun



From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Simon Cooper
Sent: Thursday, 10 August 2017 8:39 AM
To: ausnog at lists.ausnog.net
Subject: Re: [AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August

Thanks guys, appreciate the feedback.

At NEXTDC we work very hard to keep critical infrastructure maintenance that impacts racks or the environment within the data hall to an absolute minimum.

However, some works are required to happen from time to time by law and one of those activities is a de-energisation and inspection of each of our major switchboards.  This is all about ensuring that the electrical system is safe, and is an unavoidable piece of work. Rightly so.

Every rack in a NEXTDC data centre has 2N power feeds offered to it, and we highly recommend that they are used for every device in the rack.  This occasional maintenance is one of the many reasons we provide this capability.  Any device using the 2N supplies (aka A & B feeds) will simply see its power swing from one feed to the other.  Any rack that is drawing no more that its contracted power level will have no issue when all devices move from A to B or vice versa.

Our notifications were made well in advance, as we want to give customers time to consider the management of these works and we don't assume that every rack is perfectly deployed:  We are here 24/7 to assist customers with rack inspections or specific circuit down tests or maybe a controlled move prior to the day (e.g. for devices that are single fed).

Also, as we work though those discussions with customers we understand the timing in relation to their businesses, and we work very hard to coordinate schedules despite each switch board supporting many different customers and customer instances across the data centre.  Occasionally that means we have to make a difficult decisions which may result in schedule changes.  When that does happen, we do not compromise our notification periods.

I'll be happy to discuss these works with anyone - probably best offline or on the phone - but trust me when I say we are doing our absolute best to keep your data centre operating safely and reliably for many years to come.

Thanks,
Simon
COO, NEXTDC

-----Original Message-----
From: nathan at nightsys.net<mailto:nathan at nightsys.net>
Sent: Thu, 10 Aug 2017 05:39:02 +1000
To: shaun at rackcentral.com<mailto:shaun at rackcentral.com>
Subject: Re: [AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August
This seems reasonable? As long as no single Data Hall has both A and B feeds removed simultaneously, that seems normal. Doing B on Hall 2 and A on Hall 3 on the same day does seem slightly adventurous, but not entirely unreasonable (mostly in terms of if something goes bad in both, staff get stretched thin remediating issues).

A and B feeds are designed to provide redundancy to equipment, allowing one feed to fail and systems to continue on the other. Anything mission critical should be able to draw from either feed as required (dual PSU), and if there is a power deficit on one feed (causing circuits to trip), that would imply the circuits are incorrectly sized?

In addition, anything mission critical with a single power supply would be on an ATS you would assume, or a pair of equivalent devices installed with one on each rail?

Things break in datacentres, planned maintenance happens sometimes, better than no maintenance at all :) The fact they are notifying you is better than not, and it being scheduled is probably still better than unscheduled :) (easier to ensure sufficient staffing to keep a close eye on things)

Nathan.

On Wed, Aug 9, 2017 at 3:43 PM, Shaun McGuane <shaun at rackcentral.com<mailto:shaun at rackcentral.com>> wrote:


Here is the entire scheduled maintenance windows for all the halls we occupy.



DataHall 2

A Feed - Tuesday 8th August 2017 07:00 UTC +10 until 16:00 UTC +10.
B Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10.



DataHall 3

A Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10
B Feed - Monday 14th August 2017 07:00 UTC +10 until 16:00 UTC +10



DataHall 4

B Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10.



·         So looks like no power on 3 data halls B feeds on the same day.

·         Data Hall 2 was scheduled for A rail on the 8th of August but has been re-scheduled with no new date of works.

·         Data Hall 3 was scheduled for A rail on the



Problem is that I have 3x separate dates on 3 separate e-mails for Data Hall 3 and I have requested a revised clarification

Just now on the above as there has already been re-schedules for Data Hall 2 and 3.



This feels like it’s going sideways and it has not even started.



Regards

Shaun





From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net<mailto:ausnog-bounces at lists.ausnog.net>] On Behalf Of Murat Sener
Sent: Wednesday, 9 August 2017 3:20 PM
To: Matthew VK3EVL <hitman at itglowz.com<mailto:hitman at itglowz.com>>; ausnog at lists.ausnog.net<mailto:ausnog at lists.ausnog.net>

Subject: Re: [AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August



We have been told only Feed B in Data hall 2 will be effected as per original schedule and Feed A is yet to be rescheduled.







[cid:image001.png at 01D311C0.13D474C0]


Murat Sener     Datacentre Manager


PREMIER Technology Solutions
P. 1300 76 76 48    F. 1300 76 76 49    M. 0450 604 350




How am I doing? Let us know!
SIMPLY CLICK ON A FACE TO GIVE FEEDBACK


GREAT


AVERAGE


BAD


[cid:image002.png at 01D311C0.13D474C0]<https://www.crewhu.com/survey.html?crewHuID=qm0Hxbzstp&employeeID=rLeIz2SkIa&type=signat&rating=5>


[cid:image003.png at 01D311C0.13D474C0]<https://www.crewhu.com/survey.html?crewHuID=qm0Hxbzstp&employeeID=rLeIz2SkIa&type=signat&rating=0>


[cid:image004.png at 01D311C0.13D474C0]<https://www.crewhu.com/survey.html?crewHuID=qm0Hxbzstp&employeeID=rLeIz2SkIa&type=signat&rating=-5>





From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Matthew VK3EVL
Sent: Wednesday, 9 August 2017 3:13 PM
To: ausnog at lists.ausnog.net<mailto:ausnog at lists.ausnog.net>
Subject: Re: [AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August



What about both feeds happening on the same day at the same time?



On 9/08/2017 2:55 PM, Shaun McGuane wrote:

HI Damien,



My apologies that’s correct – but isn’t anyone concerned about the works being completed during business hours?

What about increased loading of the active rail once one of them is switched off for these works?



Regards

Shaun





From: Damien Gardner Jnr [mailto:rendrag at rendrag.net]
Sent: Wednesday, 9 August 2017 2:39 PM
To: <ausnog at lists.ausnog.net><mailto:ausnog at lists.ausnog.net> <ausnog at lists.ausnog.net><mailto:ausnog at lists.ausnog.net>; Shaun McGuane <shaun at rackcentral.com><mailto:shaun at rackcentral.com>
Subject: Re: [AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August



It says 7am -> 4pm in what you quoted? (UTC+10 is AEST)



On Wed, 9 Aug 2017 at 2:36 pm, Shaun McGuane <shaun at rackcentral.com<mailto:shaun at rackcentral.com>> wrote:

HI Noggers,



Has anyone seen this scheduled maintenance notification happening next week?

They maybe upgrading feeds or boards throughout the datacentre – has everyone seen this notification of works?

It appears to be happening starting 5PM through to 2AM AEST for all Datahalls on the same date.





WHAT:
NEXTDC M1 Melbourne data centre, with the assistance of specialist contractors, will conduct offline preventative electrical maintenance work on high-voltage and low-voltage main isolators. This will affect A and B feeds to the racks in Data Hall 2/3/4 on the 15th of August.

WHEN:
A Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10.
B Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10.

HOW AM I AFFECTED?
Due to the nature of the work a single feed will be removed as per the dates and times above. We recommend that customers balance the power between both power rails to minimise impact during this period. If you have single feed devices we recommend you:



Regards

Shaun







_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog

--

Damien Gardner Jnr
VK2TDG. Dip EE. GradIEAust
rendrag at rendrag.net<mailto:rendrag at rendrag.net> -  http://www.rendrag.net/
--
We rode on the winds of the rising storm,
 We ran to the sounds of thunder.
We danced among the lightning bolts,
 and tore the world asunder



_______________________________________________

AusNOG mailing list

AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>

http://lists.ausnog.net/mailman/listinfo/ausnog



MESSAGE PROTECTED BY PREMIER TECHNOLOGY SOLUTIONS<http://www.premiertech.com.au> - POWERED BY MAILGUARD<http://www.mailguard.com.au/mg>.
Report this message as spam<https://console.mailguard.com.au/ras/1RqQHQIE5J/6DWUZjiMDnlts7JYyCLNB3/0>


MESSAGE PROTECTED BY PREMIER TECHNOLOGY SOLUTIONS<http://www.premiertech.com.au> - POWERED BY MAILGUARD<http://www.mailguard.com.au/mg>.


_______________________________________________
AusNOG mailing list
AusNOG at lists.ausnog.net<mailto:AusNOG at lists.ausnog.net>
http://lists.ausnog.net/mailman/listinfo/ausnog

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20170810/47ddb0cc/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 3738 bytes
Desc: image001.png
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20170810/47ddb0cc/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 3778 bytes
Desc: image002.png
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20170810/47ddb0cc/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 4714 bytes
Desc: image003.png
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20170810/47ddb0cc/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 3771 bytes
Desc: image004.png
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20170810/47ddb0cc/attachment-0003.png>

More information about the AusNOG mailing list