Cerebras serves Qwen 3.8 27B at 1500 tokens/s, and the agent stops waiting

The docs say 1500 tokens per second

Recorded on 3 September 2026 on the Models page of the Cerebras inference documentation, the table of public models runs to two rows. Qwen 3.8 27B, id qwen-3.8-27b, 27 billion parameters, a 64k-token context window on the free tier and 128k on the paid one, speed given as roughly 1500 tokens per second. One row up, gpt-oss-120b: 120 billion parameters, 65k / 131k, and roughly 3000 tokens per second. The model four times the size runs twice as fast, which tells you plainly enough that the number is not describing the model. It is describing the machine underneath it.

What changes when the reader is an agent

On a chatbot, speed is a comfort feature. You read at a few dozen tokens per second, and everything above that lands on the floor. In an agent loop nobody is reading. The model emits a tool call, takes the result, goes round again, and does that ten or fifteen times before it hands you anything you would call useful. Every turn is text written and binned in the same breath, and the wait stacks up on each iteration.

At 1500 tokens per second, a 2000-token step is done in a little over a second. The gap between an agent that takes three minutes and one that takes twenty is not cosmetic: in the first case you stay at the desk, in the second you open another tab, and the tool has lost you.

A model served whole, which is worth saying out loud

The same page gives an entire section to how compressed its models are, and that may be the most useful thing on it if you are shopping between providers. Cerebras states that its public endpoints serve only the original, un-pruned versions. Quantisation is described as weight-only and for storage, at 16, 8 or 4 bits, with sensitive layers kept at full precision and dequantised on the fly; activations, attention and the KV cache stay unquantised.

Its models pruned with the REAP technique do exist, but on Hugging Face, for research, and not behind the API. The page also commits to not altering an existing model’s architecture without notice, and to naming any future pruned endpoints explicitly. It is the kind of promise you can only check in hindsight, but at least somebody wrote it down.

What the page does not say

Nothing about the GPU across the aisle. No comparison, no measurement protocol, no word on what exactly is being timed — both figures carry a tilde, which is already a form of honesty. Nothing on time to first token either, which matters as much as throughput inside an agent loop, and nothing on answer quality, which no tokens-per-second figure has ever measured. And nothing about the silicon: the docs are selling a throughput, not an architecture. Both models are reachable on a free trial and on usage-based pricing, subject to rate limits; anything past that goes through dedicated endpoints, which is to say through a salesperson.

Which leaves the question the documentation never asks: does an agent twice as fast buy you time, or just get you to the conclusion that your prompt was bad twice as quickly?