SambaNova and Intel Demo Air-Cooled AI Inference Split Across Two Chips
SambaNova and Intel ran GLM-5.3 on an air-cooled inference system at SC 2026: Intel D-Series GPUs handle prefill, SambaNova RDUs decode, connected by standard Ethernet.
By Nathan Brooks
3 min read

What's News
- SambaNova and Intel demonstrated an air-cooled inference system at SuperComputing 2026, announced November 15, 2026.
- Intel's D270 GPU draws 350 W air-cooled with up to 480 GB LPDDR5X per card; SambaNova's reference 80-GPU rack delivers 208 petaflops peak FP4.
- Each SambaRack SN50 holds 16 RDUs at roughly 20 kW, scaling to 256 RDUs across 16 racks as one unit.
- The system connects via NVIDIA Dynamo, vLLM and NIXL over standard RoCEv2 Ethernet — no proprietary interconnect.
- No measured latency or throughput results were reported for the combined demonstration.
SambaNova Systems and Intel demonstrated an air-cooled AI inference system at SuperComputing 2026, running GLM-5.3 across four Intel GPU D-Series accelerators for prompt processing and a SambaRack SN50 for token generation. The announcement, made November 15, splits inference into two phases and assigns each to the hardware best suited to it.
The logic is straightforward. Prefill — the compute-heavy phase that processes an incoming prompt and builds its key-value (KV) cache — runs on Intel GPUs. Decode, the memory-bound phase that generates output tokens, runs on SambaNova's fifth-generation Reconfigurable Dataflow Units (RDUs). SambaNova presents the recorded system as a design customers will be able to deploy in existing brownfield data centers.
What does this change for operators?
It removes the liquid-cooling constraint that keeps most high-end inference hardware out of older facilities. Intel's D270 is a 350 W air-cooled GPU with 32 Xe3P cores, 256 XMX matrix engines, standard PCIe 5.0 and up to 480 GB of LPDDR5X memory per card. The standard form factor is intended to work in existing servers without proprietary networking or bespoke systems. Combined with SN50's rack-level power profile, the design is built to add inference capacity within established air-cooled environments.
The scale difference matters too. Each SambaRack SN50 contains 16 RDUs and typically draws 20 kW for inference. Up to 16 racks — that's 256 RDUs — can operate as one unit, spreading a workload across lower-power racks. SambaNova says its dataflow architecture chains model operations across chips instead of returning data to memory between separate GPU kernels, cutting memory traffic and kernel-relaunch overhead during token generation.
One caveat for anyone benchmarking against this demo: it's much smaller than Intel's reference configuration. SambaNova describes an 80-GPU D-Series reference rack delivering 208 petaflops of peak FP4 compute, with 12 TB to 38 TB of memory depending on configuration. Those figures specify the larger prefill design — they are not performance measurements from the SuperComputing setup.
How do the two hardware pools talk?
No proprietary interconnect. NVIDIA Dynamo sits in front of the two accelerator pools and routes each request toward the hardware already holding its cached context. vLLM serves both the Intel prefill path and the SN50 decode path, while NIXL transfers KV-cache data over UCX/RDMA using standard RoCEv2 Ethernet. The handoff can therefore run on an established Ethernet fabric rather than a bespoke link between hardware types.
The caching layer is where the economics live. SN50 retains KV prefixes from earlier prompts in attached DDR memory, while on-chip SRAM and HBM support active decoding. When an agent returns to a conversation or shared context, cache-aware routing sends the request to the SN50 holding that prefix — so only new material goes through GPU prefill and less KV state crosses the network. SambaNova says these cache hits shorten time to first token and free GPU capacity for new requests.
Worth noting: the company describes those benefits without reporting measured latency or throughput results for the combined demonstration. Buyers will want production numbers before signing.
Independent scaling of each phase
The split also lets providers expand prefill and decode independently. Longer prompts call for more D-Series prefill capacity; agents producing longer outputs need additional SN50 decode capacity — without proportionally expanding the other pool. For an operator with fixed power and cooling budgets, that's a meaningful lever: you buy capacity where your workload actually bottlenecks.
The demonstration's significance is the integration itself. Two accelerator architectures, connected by commonly used serving software and standard Ethernet, can be assigned to different inference phases and scaled according to workload. For providers facing fixed power and cooling limits, the remaining question is whether production workloads realize enough of the described caching and utilization benefits to justify operating both pools.
Source: unite.ai
More from Nathan Brooks
Show full bio
News editor covering regulation, policy and their impact on entrepreneurship, startups and venture funding at Business Bearings.
2,232 articles