Colo Power Layout?

[jbsegal]

Might as well ask here, too.

Taken from a conversation on IRC (not #e) today. Most of this is me, with 2 other voices jumping in.

Does anyone see a downside to this idea?

JB
= = =
So, I've got this question about data center rack layout... We've got racks with 2 circuits in them (10 outlets on one, 9 on the other (after you take out outlet for the rack fan). We've got stacks of machines with dual (redundant) power supplies. Most of the machines are powered with a Y-cable going to both power supplies.

Can you see where this is going? ...

Is there any reason any of you can think of to NOT connect each Y power cable to 2 different machines, and then to make sure that each machine is powered off of 2 different circuits? That way - by my figuring - you'd have to both lose a power supply AND lose the opposite circuit for a machine to go down.

(I also have this thought that there might be some advantage to hooking the 1st power to machines A and B, the 2nd to B and C, the 3rd to C and D and so on, to the limit of the Y to reach back to A... But I'm not near so certain about that.)

someone else> usually have machines with pairs of PSUs connected to disparate power sources

Right. The Y cables (long Dell's standard choice which you had to deselect while ordering) give you HW redundancy but leave you at the mercy of your power source.

As well, as we have 2 racks, 38 outlets and 22 dual-power-supplied machines, we kinda HAVE to go with Y power.
(and something like 1/3 to 1/2 of the racks are still empty...)

So if I go with my concept, we can hook up 22 machines to 22 outlets and STILL have each of them powered off of multiple PDUs.

I don't see the lose, but I've never thought of this before.

2nd Someone> if you have remote power controllers, there is a new issue ;)

Remote power is one of the reasons to do the ab/bc/cd/da thing. I could turn off any 2 to get a machine cycled.

Comments

[ckd]

That certainly seems logical enough, given the constraints that require Y-cabling. It should give you NSPOF for power.

If the ab/bc/cd/da 4-machine cycle works, great; I suspect it may be easier to hook things up slightly differently, assuming you have enough outlets per rack:

Hook a1 up to its own outlet. Y-cable a2 and b2, b1 and c1, c2 and d2, all the way down. Hook z2 up to its own outlet.

Net requirement: N + 1 outlets for N machines, meaning with 19 outlets you can do 18 machines in a rack. Each machine on a unique pair of outlets for remote power. No stretched Y cables.

(Assuming these are 2U machines and a 40U or so rack, that's pretty close to fully packed, and I'm ignoring space for fans, PDUs, remote power controllers, KVMs, whatever...though some of those would eat up outlets too.)

[warlord-mit.livejournal.com]

Just be careful about power phase. It's possible that your two different circuits are on different phases on power. Just be sure your redundant power supplies can actually handle the multi-phase power. It's quite possible that there's no issue at all, but it's also possible that the redundancy makes some assumption on the various HOT leads coming in. YMMV. Just something to think about.

[marmota.livejournal.com]

Sounds clever to me, but as cords themselves are also potential points of failure, doubling them up could mean a worst-case scenario of downing three machines by powering off two outlets. Should be ok if the equipment in question reports how many (and even better, which) power supply(ies) it's currently running on so you can check first.

Additional considerations

[jlw.livejournal.com]

Sometimes I'll plug one side into the UPS, and the other into utility power. UPSen never, ever
fail, do they?

Then there's the layer-2 equipment that has three power supplies, but requires at least two of them energized to get enough to power all the cards in the box. This means you can't do the dual power thing because someday the one that two are on will fail leaving insufficient power. For full redundancy on these you need two separate UPS circuits (or three if you don't use utility power).

hrm...

[(Anonymous)]

seems reasonable to me...

[(Anonymous)]

Hey JB,

It's been what 10-12 years? greetings!

Did you think about a power loop that you may be causing between circuit A and circuit B if both are attached to that same Y-cable?

draw a picture. 
             pb1
              |--a.1------host T
cct A         |
==============|--a.2---)---host A
              |    /
                  /
                 /b.1
cct B           /
==============|/
              |
             pb2

now cct A and cct B are electrically connected. Do you think power will travel along b.1 to *both* host A and host T via a.2 (to host T through a.1 ) if cct A is powered off?

I would guess it depends on whether pb1 is a bus-type powerbar or if pb1 (a.1, a.2) are actually protected separate PDU sockets.

I would wonder if the load rating on cctB can handle both the bank of intended hosts plus host T (and its mates, the unintened hosts). That says nothing of the power quality issues or switching power supply interferance.

Your thoughts?

long lost Nott,
Ontario Canada via much_ado