Michael Dell announced today that Dell has delivered the world's first production NVIDIA Vera Rubin NVL72 racks to CoreWeave. That is a meaningful milestone for anyone building serious AI infrastructure. The NVL72 is not just a bigger GPU box; it is a single system with 72 Blackwell-class GPUs and NVLink-scale memory, designed for the next wave of training and inference. If I had access to that kind of capacity, I would split it cleanly in half: one side for training, the other for a large, heterogeneous inference fleet.
The split: training on one half, inference on the other
Half the rack capacity would go to training and continued pre-training. That work benefits from tightly coupled scale: large batches, high-bandwidth all-reduce, and the unified memory address space that NVL72-style systems provide. The other half I would carve into an inference pool that can serve many model variants and sizes at once.
On the inference side I would run roughly 8-10 instances of the Kimi 2.7-code model and 8-12 instances of Kimi K3. The exact number depends on context-length settings, batching strategy, and the KV-cache footprint per request, but the goal is the same: enough parallel model workers that no single hot model becomes the bottleneck. With NVL72-scale interconnect, you can place larger models in memory without the usual sharding gymnastics, and you can route requests to whichever replica has the shortest queue.
Routing: TensorZero plus Switchyard
For the gateway layer I would use two pieces of open infrastructure that I already run in production: TensorZero as the AI gateway and NVIDIA Switchyard for inference orchestration. I choose them for the same reasons I recommend them to clients: I know how they work, I trust the people maintaining them, and I know how to make them dance.
TensorZero handles model routing, observability, and cost-aware fallbacks. Switchyard manages the lower-level dispatch across the inference fleet, including prefix-aware scheduling and KV-cache-aware placement. Together they let the fleet act like one big engine while keeping each request on the cheapest, fastest path.
The operating system layer: RegicideOS
Underneath I would run our own RegicideOS image, in either the Gentoo or Arch variant depending on how much time I had to tune. RegicideOS gives an immutable root filesystem, SBOM-secured packages, and a proven NVIDIA driver installation path. On a machine this expensive, you do not want surprises from a hand-rolled OS or an untracked driver stack. Immutable roots mean you can re-image a node in minutes if a supply-chain or driver issue shows up.
Why this stack, and what it would actually do
The result is a unified but composable training-and-inference surface. The training half pushes out new checkpoints and fine-tunes. The inference half serves code-generation and general-purpose models through TensorZero and Switchyard, with RegicideOS keeping the base image reproducible. Metrics, logs, and traces flow through the same observability stack so that cost and quality can be optimized together.
This is not a theoretical exercise. We already operate pieces of this exact architecture for clients, and we use RegicideOS, TensorZero, and GPU fleet management ourselves. The difference with Vera Rubin NVL72s is the scale: instead of stitching together smaller nodes, you get to think in terms of whole model replicas and unified memory domains.
If you are lucky enough to be standing in front of a rack of these machines, the question is not whether you can run big models. You can. The question is whether your routing, OS, and cost discipline are good enough to use the hardware well. That is the part we can help with. Book a strategy call if you want to map this kind of fleet architecture to your workload.