AMD Introduces Helion Rack Blueprint
AMD has introduced Helion, an open blueprint for data center racks that aims to give CPUs a larger role in running AI agents while allowing customers to configure GPU resources around their inference requirements. The design reflects a broader shift in enterprise AI infrastructure, where the workload is no longer limited to model training or single-turn inference. Agentic AI systems combine planning, tool use, code execution, memory retrieval, workflow orchestration and repeated model calls. Those functions place different demands on computing hardware, and AMD is positioning Helion as a way to balance CPUs and accelerators inside a rack-scale architecture.
Based on Open Compute Project (OCP) specifications, the architecture supports different combinations of computing hardware. AMD cited examples including CPU-only racks for hyperscalers, powered by sixth-generation EPYC processors with up to 32,768 cores, and all-in-one enterprise racks accommodating up to 16 Instinct MI350P GPUs. The company has listed networking bandwidth of up to 102 Tbps for Helion, although it has not specified how that figure is calculated. The specification represents a planned rack capability rather than demonstrated performance in customer environments.
An integrated token router is intended to direct AI agents' requests to different models, allowing enterprises to use cloud-hosted frontier models alongside open-source models running within the rack. AMD said the design supports both air and liquid cooling. The company plans to contribute the Helion specification to OCP for further development by the industry. The chipmaker has published statements of support from infrastructure vendors, including Cisco, Dell Technologies, HPE, and Lenovo. Rack configurations developed with ecosystem partners are expected to become available in November 2026.
Why Agentic AI Could Change Server Design
Agentic AI describes systems that can pursue goals with limited human intervention. An agent might receive a request, break it into tasks, call external tools, query databases, execute code, review results and decide whether to continue or stop. Large language models and other neural networks handle the inference steps, but CPUs coordinate the surrounding process. They manage scheduling, memory access, security policies, API calls, data movement and the logic that determines which model or service should be used at each stage.
That division of labor matters because AI infrastructure has been optimized mainly around GPU throughput. Training clusters and inference farms often pair large numbers of accelerators with relatively modest CPU counts. As agentic workloads expand, the ratio may change. Lian Jye Su, chief analyst at Omdia, said the AI industry is already expecting a significant adjustment of the XPU to CPU ratio, going from 16:1 to 4:1 and eventually 1:1. XPU is an umbrella term for accelerators including GPUs and custom AI silicon.
If that forecast proves accurate, data center operators will need to rethink rack design. More CPUs per rack could improve the execution of agent logic, but it could also change power delivery, cooling requirements and network topology. AMD is not alone in preparing for that possibility. Nvidia, the dominant supplier of AI accelerators, is also expanding its use of open infrastructure standards. In the same week as AMD's Helion announcement, Nvidia said it had contributed Vera Rubin rack and compute tray designs to OCP and outlined a Vera CPU rack architecture intended to support agentic AI workloads.
The competitive context is important. Nvidia's hardware and software ecosystem has become the default platform for AI training and inference. AMD has made gains in server CPUs with its EPYC line and has built a family of Instinct accelerators, but its ROCm software stack is still fighting for broader developer adoption. Helion gives AMD a rack-level story that combines its CPU strength, its accelerator roadmap and an open-standard approach. The company is betting that some customers want more choice than a single-vendor architecture provides.
Helion Configurations and Token Routing
The Helion blueprint is modular by design. At one end of the spectrum, a hyperscaler could deploy CPU-only racks for workloads that orchestrate agents but rely on separate GPU clusters for model inference. At the other end, an enterprise could choose an all-in-one rack that mixes CPUs and up to 16 Instinct MI350P GPUs. Between those extremes, AMD says customers can configure compute, networking and cooling based on their inference and agent execution requirements.
The integrated token router is one of the more distinctive elements. In an agentic workflow, a request may move between a cloud-hosted frontier model, a smaller open-source model running on local hardware and a specialized model for a particular task. The token router is intended to direct those requests to the appropriate model. That could allow enterprises to keep sensitive data on premises while using external models for tasks that require greater general capability. It could also help operators match cost and latency to the value of each request.
Networking is another critical part of the design. AMD has listed bandwidth of up to 102 Tbps for Helion, but the company has not explained how that number is calculated or what configuration would achieve it. Even so, the figure signals that rack-scale AI systems are becoming network-centric. Agents may generate many small requests, tool calls and data exchanges. If CPUs and GPUs are separated across racks, east-west traffic can rise sharply. Low-latency networking, data locality and composable rack architectures become more important in that scenario.
Cooling is also a defining issue. Helion supports both air and liquid cooling, which gives operators options as power densities rise. Liquid cooling is increasingly used in AI data centers because accelerators and high-core-count CPUs produce large amounts of heat. An open blueprint that accommodates multiple cooling approaches could make it easier for vendors to build systems for different facility constraints.
Analysts See Existing Split, New Pressures
AMD's approach reflects an existing division of work in enterprise AI rather than a fundamental change in how AI applications are designed, according to Kanishka Chauhan, senior principal analyst at Gartner. CPUs already handle much of the orchestration surrounding GPU-based model inference, he said. The shift is one of degree rather than kind. Agentic AI may simply increase the amount of CPU work that surrounds each model call.
Charlie Dai, principal analyst at Forrester, said separating agent execution from inference would allow data center operators to scale CPU and GPU resources independently, potentially improving accelerator utilization. GPUs are expensive and power-hungry, so keeping them busy with inference is a priority. If CPUs can handle more of the orchestration, accelerators may spend less time waiting on tool calls, data retrieval or workflow decisions. The tradeoff is greater east-west network traffic, making low-latency networking, data locality, and composable rack architectures increasingly important, Dai said.
Chauhan said repeated exchanges between CPU and GPU systems could add network delays at successive stages of lengthy agent workflows, making inter-rack latency an important consideration. An agent that makes dozens of model calls, each with a round trip between a CPU rack and a GPU rack, could accumulate latency quickly. That risk may push operators toward tightly coupled designs for latency-sensitive workloads, even if disaggregated designs offer better utilization for other tasks.
Integration and Total Cost Questions
Helion could reduce the complexity of designing AI infrastructure, but enterprises would still need to calculate the total cost of ownership for individual agent workloads. Dai said potential savings from better accelerator utilization would need to be weighed against power, cooling, and integration costs. A rack blueprint may simplify procurement, but it does not remove the need to model utilization, network load and facility limits. If separating agent execution from GPU inference requires network upgrades to handle increased traffic between racks, the additional costs could offset some of the savings from more efficient GPU use.
Chauhan said Helion's all-in-one configuration could reduce the need for custom integration between CPU servers and separate GPU clusters, potentially lowering deployment costs. Hyperscalers could also deploy CPU-only racks alongside their existing GPU infrastructure, he said. That flexibility could appeal to cloud providers that already operate large GPU fleets and want to add agent orchestration capacity without replacing their accelerator investments.
Su identified software maturity, compatibility with AMD's ROCm ecosystem, and achieving operational simplicity comparable to more tightly integrated platforms as additional deployment challenges. Hardware specifications are only part of the adoption equation. Enterprises need validated designs, management tools, security controls and a software stack that developers can use without excessive friction. The statements of support from Cisco, Dell Technologies, HPE, and Lenovo are intended to address that side of the market by giving customers familiar integration and support channels.
An Alternative to Nvidia's Approach
Helion's modular design could appeal to enterprises looking to integrate AI infrastructure into existing environments. Su said an open architecture can meaningfully challenge Nvidia in segments where customers prioritize multi-vendor choice, reduced lock-in, cost flexibility, or integration with existing x86 and OCP environments. Those priorities are common among large enterprises, cloud providers and service providers that do not want their AI roadmap tied to a single supplier.
Nvidia is also expanding its use of open infrastructure standards. Its contribution of Vera Rubin rack and compute tray designs to OCP, along with a Vera CPU rack architecture for agentic AI workloads, shows that the entire industry is moving toward rack-scale reference designs. The competition is therefore not simply about chips. It is about which company can assemble the most complete ecosystem of CPUs, accelerators, networking, software, cooling and partner systems.
Dai said support from infrastructure vendors such
Source: Network World News