AI clusters have become dependent on two separate networking domains: scale-up networking, which connects accelerators inside a rack, and scale-out networking, which links racks across a data center. A single AI workload can span both domains, yet many organizations still design, operate and troubleshoot them as separate networks. Upscale, a Santa Clara, Calif.-based startup backed by Nvidia, is trying to change that with Token Fabric, an integrated architecture that combines its own scale-up silicon, Nvidia Spectrum-X-based scale-out systems and a common software layer.

Upscale introduced Token Fabric with general availability planned for early 2027. Early-access and joint-validation programs are already running. The company said the architecture is intended to give AI operators one control and observability plane across both networking tiers, rather than forcing them to stitch together separate fabrics as accelerator clusters grow from a few racks to hundreds or thousands.

Key facts at a glance

• Upscale has launched Token Fabric, an integrated scale-up and scale-out networking architecture for AI clusters.
• The design combines SkyFabriX scale-up switch silicon, scale-out systems built on Nvidia Spectrum-X Ethernet and a common software stack led by SkyOS and SkyCMD.
• General availability is planned for early 2027; early-access and joint-validation programs are running now.
• SkyFabriX supports ESUN and standard Ethernet/IP, is designed for UALink, offers 115.2 Tbps switch capacity and has a roadmap to multiple petabits per second.
• Upscale emerged from stealth in September 2025 with a $100 million seed round, then raised $190 million in June; Nvidia joined that round.
• The company has raised about $500 million, has more than 300 employees and is based in Santa Clara, Calif.
• Upscale is positioning tokens, not just packets, as a key metric for AI network performance.

Why AI networking needs a different approach

AI infrastructure has traditionally been described in terms of compute: GPUs, accelerators, memory bandwidth and power. But as clusters scale, networking has become a first-order constraint. Training and inference jobs are distributed across many accelerators, and their performance depends on how quickly data and synchronization messages move between those devices. Scale-up fabrics handle the high-bandwidth, low-latency communication inside a rack. Scale-out fabrics handle communication among racks. When those two fabrics are managed separately, operators can face inconsistent policies, fragmented telemetry and slower troubleshooting.

Barun Kar, CEO of Upscale, framed the problem as a widening mismatch between compute and networking. Compute is ramping up on a rapid scale, he said, while networking has fallen way behind. Token Fabric is Upscale's answer: a vertically integrated portfolio that spans silicon, systems and software rather than a single switch or a single operating system.

Inside Token Fabric: silicon, systems and software

Token Fabric is built from several components. At the hardware level, SkyFabriX is Upscale's scale-up switch silicon. It supports ESUN, or Ethernet for Scale-Up Networking, as well as standard Ethernet/IP. It is designed for evolving protocols such as UALink, the Ultra Accelerator Link. Upscale said SkyFabriX offers 115.2 Tbps of switch capacity, with a roadmap to multiple petabits per second.

The company also offers scale-up switch trays in custom and standard rack form factors. Those trays pool GPUs and other accelerators into large scale-up domains. For scale-out, Upscale builds systems around Nvidia Spectrum-X silicon at speeds ranging from 400G and 800G up to 1.6T. With SkyOS, the company said, those systems integrate into Nvidia and other accelerator clusters.

SkyOS is the network operating system for both fabrics. It is based on SONiC, the Software for Open Networking in the Cloud, and is optimized for AI protocols. SkyCMD is the orchestration and observability layer. It provides a single orchestration plane across scale-up and scale-out, which is central to Upscale's pitch. Instead of requiring network teams to operate one stack for in-rack accelerator communication and another for data-center-wide communication, Token Fabric aims to present one operational model.

Upscale is offering several consumption models. Customers can adopt silicon, systems, software or the full stack. The full-stack option targets neoclouds and enterprises that do not have hyperscaler-sized network engineering teams. An integrated-systems option targets XPU vendors, meaning makers of non-GPU accelerators that want validated hardware without building a complete network stack. A third option provides silicon or custom systems with a software development kit for hyperscalers that run their own network operating system. All three options draw on the same portfolio of silicon, systems and software.

Everything has to be integrated, Kar said, adding that this is where Upscale comes in and that the company is vertically stacked. The strategy reflects a broader reality in AI infrastructure: many customers want validated building blocks, but they do not all want the same level of integration. Hyperscalers may insist on controlling their own software stack, while neoclouds and enterprises may prefer a more complete system. XPU vendors need networking that works with their accelerators but may not want to become networking companies themselves.

Protocols and open-source contributions

Token Fabric is built from the protocol layer up. Upscale is extending existing protocols rather than replacing them. Scale-up traffic uses ESUN and standard Ethernet/IP. Scale-out traffic uses standard Ethernet with RoCE, which carries remote direct memory access over Ethernet. That choice matters because Ethernet is already the dominant foundation in many data centers, and RoCE has become a common way to deliver low-latency, high-throughput communication for AI and storage workloads.

Upscale is also contributing to open projects underneath the stack. Those include SONiC, extensions of the Switch Abstraction Interface, or SAI, for ESUN, the Ultra Ethernet Consortium, or UEC, and UALink. The involvement in standards and open-source communities is significant because AI networking is still evolving. Customers do not want to be locked into a proprietary protocol path if the industry converges on different approaches for scale-up or scale-out. By supporting ESUN, Ethernet/IP, RoCE and UALink while contributing to SONiC and SAI, Upscale is trying to position Token Fabric as compatible with both current and emerging designs.

Software ties the two fabrics together. SkyOS abstracts the underlying hardware and provides a control plane across the cluster. SkyCMD exposes that control plane through one layer. Multiple network elements sit behind it, and any compute platform in a heterogeneous cluster can use it to control the network. Kar described the operating system as needing to be lean and mean to be fast, flexible and secure, with an orchestration layer on top to enable capabilities such as heterogeneous compute. In practice, that means a cluster mixing different accelerator types or vendor platforms could use a common network control model rather than separate tools for each silo.

Tokens as the new networking metric

Network teams have traditionally tracked packets, throughput, loss and jitter. Those metrics remain important, but AI operators increasingly care about tokens. Tokens are the units that AI models generate and process, and they are also the basis for many commercial AI services. Kar argued that tokens are the right unit of optimization. Users pay based on tokens today, he noted, and the relevant measures include time to first token, tokens per second, tokens per dollar and tokens per watt.

Optimizing those measures requires networking, especially in clusters that scale to hundreds of thousands of accelerators. A delay in moving data between accelerators can increase time to first token or reduce tokens per second. Inefficient communication can raise cost per token or power per token. The challenge is that many networks today were not designed with token-level operations as the primary target. Kar identified latency, bandwidth and scale-up features in silicon as major components of the gap. On the software side, he pointed to predictive analytics and telemetry as tools that keep machines running.

The other side of the challenge is optimizing networking with agentic AI. Upscale has built agent-based operations directly into the platform. Kar said an agent is always looking at the network, trying to figure out whether cables will fail, optics will fail or congestion will appear, and the agents are looking for ways around those problems. That vision moves network operations from static dashboards and manual runbooks toward continuous, automated monitoring and remediation. For AI clusters, where downtime and congestion can directly affect token throughput and cost, that kind of closed-loop operations capability could become as important as raw switch capacity.

Token Fabric's early-access and joint-validation programs are intended to test those claims with customers before general availability in early 2027. The architecture also illustrates a wider shift in AI infrastructure: networking is no longer a supporting utility but a core part of how compute is pooled, scheduled and monetized. As accelerators become more powerful and clusters grow larger, the boundary between scale-up and scale-out will matter less to the application and more to the operator. Upscale's bet is that customers will want one fabric, one software layer and one operational view, even if the underlying silicon and protocols remain diverse.

Source: Network World News