[AusNOG] Data Suburb

Thu Jan 5 15:12:32 EST 2012

IMHO Bevan hit the nail on the head when talking about colo vs dedicated DC's.

ASHRAE may well support a temperature window of 27 degrees, but from a colocation perspective this is virtually impossible to meet due to different client requirements.

Similarly as Macca mentioned PUE is very much a tool to benchmark a facility against itself and can be used to further improve efficiency. It's not realistic to compare <1MW to 100MW using this metric for the reasons B outlined.

Once/if PUE2 (http://alturl.com/ztjg4 ) gets more widespread, PUE will be far more effective as a comparison tool as at the moment, there is no indication if the PUE measurement is given on the coldest day of winter with full free cooling, or averaged across the year or similar. The argument could be made that PUE should be measured worst case which I am very confident virtually all advertising claims about PUE are not on this basis.

IMO 1.24 in a desert with this scale is an excellent result.

Kind Regards
Adam Gardner
GM Operations - Vocus Data Centres
D | +61 8 9216 6623
M | +61 433 955 399
Skype| agardner.perthix
E | adam.gardner at vocus.com.au<mailto:adam.gardner at vocus.com.au>

[Description: Description: Description: Description: cid:image001.png at 01CC0F0C.40C24740]

From: ausnog-bounces at lists.ausnog.net [mailto:ausnog-bounces at lists.ausnog.net] On Behalf Of Bevan Slattery
Sent: Thursday, 5 January 2012 9:12 AM
To: Tony de Francesco; ausnog at ausnog.net
Subject: Re: [AusNOG] Data Suburb

Ok Tony.  I'll bite.

When building a 100MW colocation facility you have a base infrastructure that is required day 1 to support the modules you are building out.  Think things like 33KV feeds (or higher), step down transformer(s), HV boards, distribution to transformers to LV.  These all have loses.  May not sound like much, but they do.  That's before you even build out your facility.  These guys (and NEXTDC for that matter) are a HV customer not an LV customer like you would be.  You wouldn't be counting your portion of the buildings HV step down costs and distribution to your panel (I'm guessing).  We count our power not after our transformers, but before.  That is because PUE counts (and our power bill) from the point of mains power from the grid, not from your most convenient and number distorting/PUE improving location.  A 300KW facility?  Well it just takes an approx. a couple of 415V 400 amp 3 phase feed from the building breakers downstairs.  300KW is trivial stuff as it's all LV.

In B1 we use statics due to certain site restrictions.  And you're right, we can include modules of 200KW at t time.  But in fact we have already fully loaded our UPS's to their 1.2MW capacity.  Why?  Because we can already see the demand coming in the next 3-6 months *and* we made the conscious decision that we will take a PUE hit early to IMPROVE reliability.  We decided with this in mind we wanted to put our A and C power rooms/UPS's dark straight away to reduce interference with systems during upgrades.  Yes things are designed so they are modular and hot swappable, but having operated data centres before, I can tell you sometimes things don't go to plan, regardless how good your planning is.  We were looking at 9 UPS upgrades over the 6 months.  Imagine being the customer receiving those notices! :)  So it's not because it's poor UPS/power design my Brisbane PUE will be a little higher earlier, it's because of good data centre management.

We made a conscious decision not to use static UPS's in our purpose built facilities where possible.  We use DRUPS which are about the same energy efficiency at full load as static UPS which provide actual protection 100% of the time rather than the more efficient eco-cycle statics.  Why?  Well we really like the fact these things are online all the time, offer outstanding power protection and can scale effectively and with great reliability, particularly in IP-Bus config.  We use Pillars' ISO Parallel Bus system which has tremendous benefits (which I'm not going to go through here).  At high load we won't get greater than 96% efficiency (less than the eco-modes of other static UPS manufacturers), but I'm old fashioned.  I like to provide my customers online protection 100% of the time, not just when there's a problem and the static exits Eco-mode and starts to actually inject itself in the power path and actually invert, which maybe too late.  We may one day go down this path, but this technology IMHO is too new to put into a colo facility in which some of our customers have told us they will not accept Eco-Mode based UPS.  Rotaries aren't as efficient as statics under partial load.  We know that.  So under partial load we will be less efficient.  But at full load the facility will be marginally better with Rotaries rather than normal statics with the benefit of not throwing out 200t of batteries every 7-10 years in order to achieve Teir III status (as we would in Melbourne).

There are cost efficiencies with operating larger rooms (to a point) from a capital and operational perspective and this has an impact.  It also provides certain flexibilities and capabilities that 300KW just can't deal with.  It also provides some other benefits in terms of communities of interest.  This is colocation land I'm talking about here.  I appreciate with containment this is reduced significantly, but losses do happen.  Thermal mass of rooms, fan static pressures still leak slightly etc..  Building a big room and big facility needs to pump water further from day one, needs to operates lifts and lighting for common areas (think security, delivery, foyers, breakout rooms, elevators etc..), needs switching equipment, needs fire panels, base building airconditioning etc. etc. etc..

You would realise this if you built 40 of your 300KW modules side by side,  which is what you'd need to do to compare your "facilities" to M1 or 300 of them to compete with SwitchNap.   Sometimes people take a higher partial PUE not because of poor design, but because of good, long term, scalable and affordable design of larger facilities and sometime adopting a more proactive approach to upgrades to provide additional reliability.

And people remember this is a colocation facility designed for the requirements of many.  I think it's one of the most efficient (if not) true colo environments in that type of region in the world.  Well it certainly was when it was built.  This isn't meant to be a stab at Google here, but building your own facility for your own use, where you are the only "customer" happy with wider temp/humidity levels and certain levels of failure due to global cloud failover is not an appropriate comparison.

Cheers

[b]

From: ausnog-bounces at lists.ausnog.net<mailto:ausnog-bounces at lists.ausnog.net> [mailto:ausnog-bounces at lists.ausnog.net]<mailto:[mailto:ausnog-bounces at lists.ausnog.net]> On Behalf Of Tony de Francesco
Sent: Thursday, 5 January 2012 9:50 AM
To: ausnog at ausnog.net<mailto:ausnog at ausnog.net>
Subject: Re: [AusNOG] Data Suburb

The use of the word "poor" was not a comment on the NevadaNAP's PUE performance and was in no way meant to detract from their achievement of a PUE of 1.24 - it was in response to a comment that PUE "takes a hit when capacity is underutilized".

This statement has permeated the industry and led to people accepting lower levels of efficiency at part load in the hope that their PUE will improve as the load increases.  There is no technical reason why PUE has to suffer at part utilization, other than people accepting a facility design that allows this.

This is true for a 300kW facility or 100MW facility - nothing disingenuous about that.

Kind Regards

Tony de Francesco
Technical Director
email: tonyd at pue.com.au<mailto:tonyd at pue.com.au>
mob: +61 (0) 457 701 179
.....................................................
[cid:image003.jpg at 01CCCBA2.B2D955F0]

The information contained in this email and any attachments may be confidential. This email and any attachments are also subject to copyright. No part of them may be reproduced, adapted or transmitted without the written permission of the copyright owner. If you are not the intended recipient, any use, interference with, disclosure or copying of this information is unauthorised and prohibited. If you have received this email in error, please immediately advise the sender by return email and delete the message from your system. All email communications to and from NEXTDC Limited are recorded for the purposes of archival and storage.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20120105/8ce28f18/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 3984 bytes
Desc: image002.png
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20120105/8ce28f18/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.jpg
Type: image/jpeg
Size: 24527 bytes
Desc: image003.jpg
URL: <http://lists.ausnog.net/pipermail/ausnog/attachments/20120105/8ce28f18/attachment.jpg>