Cerebras reports 5× inference throughput per megawatt with GPUs
Cerebras Systems says a production setup using AMD Instinct GPUs for prefill delivered 5× throughput per megawatt, without reducing token speed, ahead of a broader 2027 rollout.
By Grace Kim
4 min read

What's News
- November 17, 2026: Cerebras reported 5× throughput per megawatt versus a CS-3-only setup.
- November 14, 2026: The production test used a shared GPT-OSS-120B endpoint and a normalized 1× baseline.
- Cerebras says its chip has 350× the SRAM capacity and 5,400× the memory bandwidth of a typical GB300 GPU.
- Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute are early-access deployment partners.
- By the end of 2027, Cerebras targets hundreds of megawatts across its cloud and partner deployments.
Cerebras Systems reported on November 17, 2026, that its production disaggregated-inference deployment delivered five times the throughput per megawatt of a CS-3-only configuration without reducing token-generation speed.
The result applies to Cerebras Inference Cloud. Cerebras measured it on a shared GPT-OSS-120B endpoint on November 14, 2026, comparing throughput per megawatt before and after disaggregation. The company normalized the aggregated configuration to a 1× baseline.
Moving prompt processing to AMD Instinct GPUs left CS-3 serving capacity available for response generation, Cerebras said. The arrangement expands the fast-inference capacity the company can offer through its cloud.
How does production testing build on the October result?
Cerebras’ October 1, 2026 explanation of disaggregated inference reported a 5× gain in early tests using the same wafer-scale-engine footprint. The November 17 announcement advances that result by adding a production measurement and a deployment package for infrastructure partners.
What changes when prefill and decode run apart?
Each inference request has two principal stages, Cerebras explained. Prefill processes the input prompt and creates intermediate model state. Decode uses that state to produce output tokens one at a time, feeding each new token back into the model.
Because every decode step reads model data from memory, Cerebras said, memory bandwidth is central to interactive generation speed. Prefill performs more computation across a prompt in parallel, which makes it a better fit for accelerators built for that load.
Cerebras’ wafer-scale systems keep model data in on-chip SRAM rather than relying primarily on off-chip high-bandwidth memory. The company says each wafer-scale chip carries tens of gigabytes of SRAM and tens of petabytes per second of memory bandwidth.
Cerebras says its SRAM capacity is 350× that of a typical GB300 GPU and its memory bandwidth is 5,400×. It said the on-chip capacity allows more concurrent requests and model layers to remain directly on the chip for rapid decode.
Cerebras’ logic is straightforward: reserve the wafer-scale system for memory-bound token generation.
Does the speed gain have a clear economic target?
For power-constrained data centers, Cerebras is targeting more customer-serving capacity and revenue potential from the same power budget. The company cited Muse, Copilot and Codex as examples of rapidly scaling agentic applications that require highly interactive responses.
Cerebras said the capacity increase must preserve generation speed to be useful for those workloads. The measured configuration did so while moving prompt work away from the CS-3 systems.
The business case is more serving output from a fixed power envelope, according to Cerebras.
How can infrastructure partners deploy the system?
Partners can use a Cerebras-provided reference architecture that co-locates their chosen prefill accelerators and Cerebras systems on a unified network, the company said. Cerebras offers two integration routes for that architecture.
- Cerebras’ end-to-end software stack, intended to shorten partners’ time to market, routes requests between prefill and decode pools and manages replica capacity and health. Its prefill engine can run open-source inference engines such as vLLM or SGLang, while Cerebras supplies the decode engine.
- A Cerebras decode-only SDK gives partners more control. They retain their own inference framework and prefill engine while using Cerebras’ decode engine and KV Portal.
Cerebras frames the choice as speed to market versus greater control over the deployment.
What moves state between the two engines?
Under Cerebras’ full stack, KV Portal transfers intermediate model state between the prefill and decode engines over RDMA, the company said. The transfer includes key-value caches that preserve request-specific context after prefill.
The decode-only SDK also includes a mock decode replica and a reference integration with an open-source disaggregated inference framework and prefill engine, Cerebras said. Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute are named as early-access partners, and the company said it is expanding access to additional partners.
Both routes rely on KV Portal to carry request context between the processing stages.
How quickly can Cerebras scale the deployment?
Cerebras plans to deploy disaggregated inference broadly across hundreds of megawatts in its own inference cloud and partner deployments by the end of 2027. Cerebras Systems described the figure as “a deployment target, not capacity already operating.”
The announcement pairs a measured cloud configuration with a software path for outside operators, according to Cerebras. Its next test is whether the throughput advantage on one shared endpoint holds across partners’ selected accelerators, frameworks and deployment sizes.
The broader rollout will show whether the 5× result can carry across different hardware, software and operating scales.
Source: unite.ai
More from Grace Kim
Show full bio
Market editor covering people, launches and day-to-day practice in entrepreneurship, startups and venture funding at Business Bearings.
2,301 articles