Traditional data center design uses raised floors, perforated tiles, rows of cabinets surrounded by hulking HVAC units and giant PDUs, and a whole room dedicated to batteries. No thanks. I have a better way.
Raised floors are poor ways to move cold air around, and then we fill the air handling space with power and network cables and wonder why we have hot spots and high fan loads. Raised floors are expensive, dangerous (ever fallen into an open tile?), not very earthquake resistant, and capable of hiding too many sins -- like dust bunnies the size of German Shepherds.
My last few data centers have worked out OK without raised floors. On the other hand, we still have hot spots, visible rat's nests of network and power cables, and the occasional dust bunny. This time will be better.
What's a good data center like without raised floors? Turns out we can steal a few ideas from good old Ma Bell central office designs:
- Everything is mounted in cabinets bolted to concrete platforms. The platforms raise all the equipment above the inevitable self-inflicted floods, without enabling the hiding of cables, dirty magazines, and other sins. In earthquake country, we'll put down seismic isolation platforms (like these).
- Power and network cables run in overhead trays or ladder racks.
- Run with wider temperature ranges, no humidity controls, and lots of fresh air ("air side economization").
Going beyond Ma Bell:
- Use overhead three-phase power busses (like these) to maximize power flexibility to each cabinet.
- Use 208V single-phase AC power whenever possible to maximize efficiency; 110 VAC if really needed. DC powered servers are still rare, and DC power operating knowledge is rare, too.
- Strict cold air/hot air isolation. Flood the room with cold air, force that air to go through IT equipment only with baffles and seals, and vent the hot air through cabinet-mounted chimneys (like these), directly back to the HVAC system or outdoors.
- With a 60 KW data center, use cabinet-mounted small UPSes, instead of a central UPS that's typically oversized and running inefficiently.
- Keep the HVAC units out of the data center as much as possible. Ideally, roof-mounted units, with big fat air ducts into the data center.
- Absolutely No Humidification Water in the Data Center. Rube Goldberg must have been lead designer of all of the humidification systems I've seen. Stopping water leaks and sprays with dozens of kilowatts of power a few inches away is no fun. Can't get rid of water entirely, I always lose the sprinkler arguments with the fire department.
More later ...