@futurebird Yes, most large datacenters have I'm sure many metric shittons of ethernet cables in them. Usually a lot of work goes into cable management, otherwise things
would be totally out of control.

There are companies that have even designed large compute clusters in clever ways to minimize cable length, which minimizes latency and saves money and effort in wiring.

There's often a fair bit of fiber optic cables too, but there's tradeoffs in terms of cost. Also, it takes an incredibly beefy server to be able to make much use of the very high bandwidths that fiber optics provide, so sometimes application servers are copper to switches, which aggregate multiple servers together into one very high bandwidth fiber optic backbone link.

@lopta @futurebird @oxidecomputer "Until Oxide, cloud architecture remained exclusively in public cloud datacenters, inaccessible to 85% of global IT workloads that run on-premises."

Huh, so I guess they're pretending OpenStack never existed?

@spacehobo @futurebird @oxidecomputer I think they're talking more about a tightly-integrated platform underneath that software stack.

@lopta @futurebird @oxidecomputer I mean, there's always k8s for that kind of thing, but maybe I'm one of those "cloud computing just means a VM spawning API and crypto is short for cryptography" Olds now.

This video pretty much shows what I expected, although finding out that there are millions of machines on those racks was kind of vertigo inducing. This is what "a data center" looks like today. And people keep all those cables tidy, and replace the servers when they break.

@spacehobo @futurebird @oxidecomputer What I've seen of their work is encouraging but I haven't got to test it myself. Looks like they put a lot of work into power, networking and firmware in an attempt to weed out a lot of the cruft that you get with racks of raditional servers.

@futurebird We recently brought in these same cable-tidiers to manage my (much, much smaller) research lab, and they are much, much better at running cables neatly than I am. It is very obvious which racks they wired and which ones I did.

(This is probably why they look nervous whenever I walk into the server room.)

@lopta @spacehobo @futurebird @oxidecomputer to me the most impressive part is where they completely replaced the BIOS with their own firmware, written from scratch in rust, that is co-designed with the kernel.

But also, close to OP's point, they designed the rack so that the owner never does any cable management within the rack. E.g. when you need to add another motherboard, you just slide it into an empty slot, and the back of it auto-connects to DC power & network cables.

@lopta @spacehobo @futurebird @oxidecomputer my mind went to oxide as well, and while I can’t find pictures they do things like integrating power and network into the rack chassis, so replacing a compute/storage sled is just a case of sliding a new one in. No running cables. I don’t know of anybody else *selling* that kind of thing.

@lopta @spacehobo @futurebird @oxidecomputer in principle you could extend that, especially in sci-fi. In the same way these take common component like power and sacrifice flexibility of compute nodes (you need to buy Oxide devices) for easy maintenance (you don’t need to recable), you could do that with the whole rack. Replace the device with blocks of, I don’t know, memistor arrays that combine memory and compute in the same components. Fill an asteroid with precisely machined tunnels; line them with superconducting rails that can be transport, power supply, or network as needed. Route around failed nodes with virtualization until you have enough to shuffle them. Or just take the hit like an SSD and overprovision so you have spare nodes to fill in. There’s enough weird future tech in the concept of a 20k year data centre that it’s hard to know what failure modes need to be considered; but if you have reliable enough hardware overprovisioning and protecting it may be a better approach than trying to build repair systems which add additional (and different) complexity. I would imagine that at that scale storage would have to be kept live; I’d expect high performance memory to decay from Weird Quantum Effects.

@futurebird not one solid machine to survive that long. delocalized, the cables are alive and constantly growing repairing re-attaching, especially as new data storage units grow..

@futurebird one solid state machine never touched by radiation, connected to the outside by connections that are oxidation proof, thermal shock proof, physical shock proof, ...

And if you put up a lead shield to block the low energy radiation, the high energy radiation becomes low(er) energy ...

If I was designing this data center, I would hope I did not intend to be alive 200 years from now ...