[datacenter] Server room build questions

[Paul English <tallpaul@speakeasy.org>]

I posted roughly the following to my local SAGE list and it was suggested 
that I also ask here. I'm excited (and intimidated) to do this since I 
figure that this sort of opportunity doesn't come along very often in a 
sysadmin's career. 

I finally have a chance to set up a server room properly and I'd love to 
get some tips and feedback on what, how and who. 

We are looking at signing a 5 year lease and as part of the lease having 
the landlord build the server room space for us (using a standard "Tenant 
Improvement" allowance). I'm keen to be very specific about what we want to 
ensure that we don't end up getting stuck. We currently have 4.5 racks that 
we've built up over the last 3 years, and I think we will be adding 1 more 
rack in the next 6 months. We are looking at getting enough space for approx 
18 racks which should be more than enough. 

We have machines in 3 categories:  

3/4 rack of 24/7 critical customer-facing systems, network switches, T1 
CSU/DSU, etc. This probably won't be growing much.

1 rack of "operational" systems which can be down for 24 hours max. These 
are growing slowly to maybe 2 racks by the end of 5 years. 

All the rest which are growing the fastest are research style computational 
nodes which can be shut down for a day or two if necessary, and individual 
nodes can be up and down at various times with no issue. This is what will 
likely grow up to 12 racks over the next 5 years. Probably less. 

So on to the questions:
Given a commercial office space, does a roof-mounted AC make the most 
sense?

What are people's feelings on raised floors vs. tiles/anti-static carpet? 
I'm very opposed to running network/kvm cables under floors (give me a 
ladder rack any day), but raised floors can be handy for routing AC to the 
front of the racks and running fat trip-causing power cables. 

How much AC should I spec - get the maximum capacity 18 rack's worth now, 
or get a lesser amount now and plan to upgrade? 

We already have a backup Movincool AC unit which will sit in the room set 
to fire up if the main AC should fail. In this case we'll shut down all 
but 1 or 2 racks of equipment.

How much power should I spec? On this one I'm fairly sure that I should 
probably spec the full 20 rack's worth (800A?) and just get the breaker 
panel put into the room with circuits sufficient for 6 racks. 

What are the considerations for a backup generator vs very large 
(Liebert or APC) UPS? I'm leaning towards an auto-start generator 
right now, with smaller front-end UPS to keep everything going while 
research systems are powered down and the generator fires up, and 
additional rack-mount UPSen for the 24/7 systems. 

Do we bother with a fancy dry fire supression system? Seems like a major 
PITA and would take a big chunk out of the budget for everything else. 
A fine mist system+ hardware insurance  + offsite web mirror has been 
suggested. Currently there is *no* fire supression installed in the space, 
only a (legal at the time) fire hose in the hallway. 

Vendor recommendations are a plus, although since the landlord will be 
footing the bill, he gets to choose. 

Thanks in advance for the feedback. I think the answers will be useful 
to everyone but if you feel like sending me your answer off-list I will 
summarize the replies.

Paul

Below is a rough draft of the specs that I will send to the landlord. It 
was written based on feedback from the local SAGE list. I realize that all 
I'm asking for below may not fit into our TI allowance (typically up to 1 
year's rent on a 5 year lease - in our case something like $50,000), and I 
will have to cut it back. Suggestions there are also useful:

Data Center Build Specifications

1.Electrical Service

800amp service with breaker panel accessible preferably installed in the 
room if code permits. Uninterruptable Power Supply with auto-starting 
generator for continuous power, maintained and tested per manufacturer 
specifications.  UPS+genset backed 2 times 110V/20 amp power outlets 
installed every 24 inches along ceiling or floor down the center of the 
room, except for the door clearance and AC blower areas.

One 220V/20amp twist-lock power UPS+genset backed power outlet for 
portable backup AC unit installed on the wall near the door (approx 60 
inches from doorway). 12 inch diameter round ducting for backup AC unit 
installed 60 inches from doorway (tenant-owned Movincool Classic Plus 26 
unit) vented to the outside, either through the roof or exterior wall.

EPO panic button installed on the wall near doorway.

2.Connectivity

     4 labelled runs of Category 5e data cable from the 3rd floor 
data/phone closet to the new data center

     4 labelled runs of Category 5e data cable from the 3rd floor office 
space to the new data center
     2 labelled runs of dual fiber optic cable terminated for 1000BaseSX  
from the 3rd floor office space to the new data center

2.Environmental Control

24 tons Precision AC maintained and tested per manufacturer 
specifications. Roof-mounted unit with blower unit installed at the far 
end of the room from the door with cold air directed down left-hand side 
of room, hot air return on right. 

Soundproofing insulation installed in walls facing other tenants.

3.Room Structure

Low voltage cable management ladder rack suspended from the ceiling, one 
run 2 feet from left wall, and one run 2 feet from right wall with 
cross-connects every 4 feet.

Anti-static non-raised tile floor. 

Double 36-inch doors at room entrance, with keyed access on one door, 
floor/ceiling throw bolts on the other.

4.Fire Suppression

Fine mist fire suppression system or FM-200 chemical fire suppression 
system.

---------------------------------------------------------------------
To unsubscribe, e-mail: datacenter-unsubscribe@shorty.com
For additional commands, e-mail: datacenter-help@shorty.com