OCP Global Summit this week showed AI infrastructure splitting along lines of labor: CPU for agent execution, GPU plus specialized accelerators for compute, shared memory, and power, cooling, and networking as design starting points.
1. AMD's Helion rack separates agent execution from model inference. The pure-CPU config scales to 32,768 cores for running agent workloads, connected to a separate inference system. Calling models and executing tasks now justify distinct infrastructure.
2. Nvidia contributed Vera Rubin rack and compute-tray designs through its MGX open architecture. That covers liquid-cooled blind-mate connectors, fanless trays, and high-current busbars for racks up to about 250kW. It also opened part of its DSX AI-factory management software.
3. SK Hynix's MARS architecture pools memory across servers instead of tying it to individual CPU/GPU nodes. In multi-turn agent inference, shared KV cache delivered a 3-5x first-token latency improvement versus recompute. Memory vendors now design how a rack organizes compute and data.
4. D-Matrix showed attention-FFN splitting between GPU and its Corsair XPU, across a system connecting up to 1,024 XPUs. Specialized architecture is entering real heterogeneous inference pipelines rather than demanding wholesale GPU replacement.
5. Google and Microsoft are co-designing from chip to grid. The grid-resilience spec calls for data centers to stay connected, respond automatically, and restore near-full load in under two seconds. Microsoft's Mt. Diablo cabinet separates power conversion from compute racks across a roughly 100kW to over 1MW modular range.
6. Lightmatter and Dell are working on how co-packaged optics become installable, testable, and maintainable products. They are defining mechanical, power, signal, thermal, and management requirements for front-panel, near-package, and co-packaged optics.