[AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August

Thu Aug 10 09:12:42 EST 2017

An off-the-cuff observation with little real thought put into it:

offloading an A feed in one hall and a B feed in the other (if at the same time) seems a reasonable method - intentional or not - of keeping the cumulative data centre load even across feeds. 

They should be fine standalone anyway, but high current loads have a way of finding any weakness in the circuit path. 

As mentioned, use redundant power/ATS whatever to ensure your critical kit up, and have faith in your choice of DC. 

Karl

> On 10/08/2017, at 10:39 AM, Simon Cooper <spgc at inbox.com> wrote:
> 
> Thanks guys, appreciate the feedback.
> 
> At NEXTDC we work very hard to keep critical infrastructure maintenance that impacts racks or the environment within the data hall to an absolute minimum.
> 
> However, some works are required to happen from time to time by law and one of those activities is a de-energisation and inspection of each of our major switchboards.  This is all about ensuring that the electrical system is safe, and is an unavoidable piece of work. Rightly so.
> 
> Every rack in a NEXTDC data centre has 2N power feeds offered to it, and we highly recommend that they are used for every device in the rack.  This occasional maintenance is one of the many reasons we provide this capability.  Any device using the 2N supplies (aka A & B feeds) will simply see its power swing from one feed to the other.  Any rack that is drawing no more that its contracted power level will have no issue when all devices move from A to B or vice versa.
> 
> Our notifications were made well in advance, as we want to give customers time to consider the management of these works and we don't assume that every rack is perfectly deployed:  We are here 24/7 to assist customers with rack inspections or specific circuit down tests or maybe a controlled move prior to the day (e.g. for devices that are single fed).
> 
> Also, as we work though those discussions with customers we understand the timing in relation to their businesses, and we work very hard to coordinate schedules despite each switch board supporting many different customers and customer instances across the data centre.  Occasionally that means we have to make a difficult decisions which may result in schedule changes.  When that does happen, we do not compromise our notification periods.
> 
> I'll be happy to discuss these works with anyone - probably best offline or on the phone - but trust me when I say we are doing our absolute best to keep your data centre operating safely and reliably for many years to come.
> 
> Thanks,
> Simon
> COO, NEXTDC
> 
> -----Original Message-----
> From: nathan at nightsys.net
> Sent: Thu, 10 Aug 2017 05:39:02 +1000
> To: shaun at rackcentral.com
> Subject: Re: [AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August
> 
> This seems reasonable? As long as no single Data Hall has both A and B feeds removed simultaneously, that seems normal. Doing B on Hall 2 and A on Hall 3 on the same day does seem slightly adventurous, but not entirely unreasonable (mostly in terms of if something goes bad in both, staff get stretched thin remediating issues).
> 
> A and B feeds are designed to provide redundancy to equipment, allowing one feed to fail and systems to continue on the other. Anything mission critical should be able to draw from either feed as required (dual PSU), and if there is a power deficit on one feed (causing circuits to trip), that would imply the circuits are incorrectly sized?
> 
> In addition, anything mission critical with a single power supply would be on an ATS you would assume, or a pair of equivalent devices installed with one on each rail?
> 
> Things break in datacentres, planned maintenance happens sometimes, better than no maintenance at all :) The fact they are notifying you is better than not, and it being scheduled is probably still better than unscheduled :) (easier to ensure sufficient staffing to keep a close eye on things)
> 
> Nathan.
> 
> On Wed, Aug 9, 2017 at 3:43 PM, Shaun McGuane <shaun at rackcentral.com <mailto:shaun at rackcentral.com>> wrote:
> Here is the entire scheduled maintenance windows for all the halls we occupy.
> 
>  
> 
> DataHall 2
> 
> A Feed - Tuesday 8th August 2017 07:00 UTC +10 until 16:00 UTC +10.
> B Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10.
> 
>  
> 
> DataHall 3
> 
> A Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10
> B Feed - Monday 14th August 2017 07:00 UTC +10 until 16:00 UTC +10
> 
>  
> 
> DataHall 4
> 
> B Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10.
> 
>  
> 
> ·         So looks like no power on 3 data halls B feeds on the same day.
> 
> ·         Data Hall 2 was scheduled for A rail on the 8th of August but has been re-scheduled with no new date of works.
> 
> ·         Data Hall 3 was scheduled for A rail on the
> 
>  
> 
> Problem is that I have 3x separate dates on 3 separate e-mails for Data Hall 3 and I have requested a revised clarification
> 
> Just now on the above as there has already been re-schedules for Data Hall 2 and 3.
> 
>  
> 
> This feels like it’s going sideways and it has not even started.
> 
>  
> 
> Regards
> 
> Shaun
> 
>  
> 
>  
> 
> From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net <mailto:ausnog-bounces at lists.ausnog.net>] On Behalf Of Murat Sener
> Sent: Wednesday, 9 August 2017 3:20 PM
> To: Matthew VK3EVL <hitman at itglowz.com <mailto:hitman at itglowz.com>>; ausnog at lists.ausnog.net <mailto:ausnog at lists.ausnog.net>
> 
> Subject: Re: [AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August
> 
>  
> 
> We have been told only Feed B in Data hall 2 will be effected as per original schedule and Feed A is yet to be rescheduled. <>
>  
> 
>  
> 
>  
> 
> <image001.png>
> 
> Murat Sener     Datacentre Manager
> 
> PREMIER Technology Solutions
> P. 1300 76 76 48    F. 1300 76 76 49    M. 0450 604 350
> 
>  
> 
> How am I doing? Let us know!
> SIMPLY CLICK ON A FACE TO GIVE FEEDBACK
> 
> GREAT
> 
> AVERAGE
> 
> BAD
> 
> <image002.png> <https://www.crewhu.com/survey.html?crewHuID=qm0Hxbzstp&employeeID=rLeIz2SkIa&type=signat&rating=5>	
> <image003.png> <https://www.crewhu.com/survey.html?crewHuID=qm0Hxbzstp&employeeID=rLeIz2SkIa&type=signat&rating=0>	
> <image004.png> <https://www.crewhu.com/survey.html?crewHuID=qm0Hxbzstp&employeeID=rLeIz2SkIa&type=signat&rating=-5>
>  
> 
> From: AusNOG [mailto:ausnog-bounces at lists.ausnog.net <mailto:ausnog-bounces at lists.ausnog.net>] On Behalf Of Matthew VK3EVL
> Sent: Wednesday, 9 August 2017 3:13 PM
> To: ausnog at lists.ausnog.net <mailto:ausnog at lists.ausnog.net>
> Subject: Re: [AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August
> 
>  
> 
> What about both feeds happening on the same day at the same time?
> 
>  
> 
> On 9/08/2017 2:55 PM, Shaun McGuane wrote:
> 
> HI Damien,
> 
>  
> 
> My apologies that’s correct – but isn’t anyone concerned about the works being completed during business hours?
> 
> What about increased loading of the active rail once one of them is switched off for these works?
> 
>  
> 
> Regards
> 
> Shaun
> 
>  
> 
>  
> 
> From: Damien Gardner Jnr [mailto:rendrag at rendrag.net <mailto:rendrag at rendrag.net>] 
> Sent: Wednesday, 9 August 2017 2:39 PM
> To: <ausnog at lists.ausnog.net> <mailto:ausnog at lists.ausnog.net> <ausnog at lists.ausnog.net> <mailto:ausnog at lists.ausnog.net>; Shaun McGuane <shaun at rackcentral.com> <mailto:shaun at rackcentral.com>
> Subject: Re: [AusNOG] NextDC Melbourne - Scheduled Power Maintenance - 15th August
> 
>  
> 
> It says 7am -> 4pm in what you quoted? (UTC+10 is AEST)
> 
>  
> 
> On Wed, 9 Aug 2017 at 2:36 pm, Shaun McGuane <shaun at rackcentral.com <mailto:shaun at rackcentral.com>> wrote:
> 
> HI Noggers,
> 
>  
> 
> Has anyone seen this scheduled maintenance notification happening next week?
> 
> They maybe upgrading feeds or boards throughout the datacentre – has everyone seen this notification of works?
> 
> It appears to be happening starting 5PM through to 2AM AEST for all Datahalls on the same date.
> 
>  
> 
>  
> 
> WHAT:
> NEXTDC M1 Melbourne data centre, with the assistance of specialist contractors, will conduct offline preventative electrical maintenance work on high-voltage and low-voltage main isolators. This will affect A and B feeds to the racks in Data Hall 2/3/4 on the 15th of August.
> 
> WHEN:
> A Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10.
> B Feed - Tuesday 15th August 2017 07:00 UTC +10 until 16:00 UTC +10.
> 
> HOW AM I AFFECTED?
> Due to the nature of the work a single feed will be removed as per the dates and times above. We recommend that customers balance the power between both power rails to minimise impact during this period. If you have single feed devices we recommend you:
> 
>  
> 
> Regards
> 
> Shaun
> 
>  
> 
>  
> 
>  
> 
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net <mailto:AusNOG at lists.ausnog.net>
> http://lists.ausnog.net/mailman/listinfo/ausnog <http://lists.ausnog.net/mailman/listinfo/ausnog>
> --
> 
> Damien Gardner Jnr
> VK2TDG. Dip EE. GradIEAust
> rendrag at rendrag.net <mailto:rendrag at rendrag.net> -  http://www.rendrag.net/ <http://www.rendrag.net/>
> --
> We rode on the winds of the rising storm,
>  We ran to the sounds of thunder.
> We danced among the lightning bolts,
>  and tore the world asunder
> 
> 
> 
> 
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net <mailto:AusNOG at lists.ausnog.net>
> http://lists.ausnog.net/mailman/listinfo/ausnog <http://lists.ausnog.net/mailman/listinfo/ausnog>
>  
> 
> 
> MESSAGE PROTECTED BY PREMIER TECHNOLOGY SOLUTIONS <http://www.premiertech.com.au/> - POWERED BY MAILGUARD <http://www.mailguard.com.au/mg>. 
> Report this message as spam <https://console.mailguard.com.au/ras/1RqQHQIE5J/6DWUZjiMDnlts7JYyCLNB3/0>  
>  
> 
> 
> MESSAGE PROTECTED BY PREMIER TECHNOLOGY SOLUTIONS <http://www.premiertech.com.au/> - POWERED BY MAILGUARD <http://www.mailguard.com.au/mg>.  
>  
> 
> 
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net <mailto:AusNOG at lists.ausnog.net>
> http://lists.ausnog.net/mailman/listinfo/ausnog <http://lists.ausnog.net/mailman/listinfo/ausnog>
> 
> 
> _______________________________________________
> AusNOG mailing list
> AusNOG at lists.ausnog.net
> http://lists.ausnog.net/mailman/listinfo/ausnog

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20170810/3114c62a/attachment-0001.html>