Nvidia's Groq 3 LPX inference chip enters full production

Nvidia announced on 24 August 2026 that Groq 3 LPX, a chip built to do nothing but generate tokens, is in full production. The bet behind it is architectural: instead of one GPU doing every phase of an inference request, Nvidia now splits the job, leaving Rubin GPUs to chew through context while a separate accelerator handles the part a user actually waits on. For anyone building agents, this is a claim about the one number that decides whether an agent feels alive or broken - time to the next token.

How the work gets split

Nvidia's developer blog describes the division precisely: Vera Rubin GPUs handle prefill and decode attention, while the LPX accelerators take the latency-sensitive decode components - feed-forward and mixture-of-experts execution - through what Nvidia calls an Attention-FFN Disaggregation loop. NVIDIA Dynamo orchestrates the heterogeneous serving across those disaggregated backends.

Why bother? The two phases of an inference request want opposite hardware. Reading a 100,000-token context is a bandwidth-bound batch job that suits a GPU. Emitting the next token is a small, serial, latency-bound operation that mostly waits on memory. Running both on the same silicon means one of them is always badly served, and in an agent loop - which emits tokens continuously while tools return more context - the badly served one is the one the user sees.

The rack, on paper

Spec	Figure
Accelerators per rack	256
FP8 compute	315 PFLOPS
Total SRAM	128 GB
On-chip bandwidth	40 PB/s
Chip-to-chip bandwidth	640 TB/s

Every figure in that table comes from a single source - Nvidia's own developer blog, published on 16 March 2026, six months before this production announcement. The same post claims up to 35 times higher inference throughput per megawatt and up to 10 times more revenue opportunity for trillion-parameter models than GB200 systems, measured at 400 tokens per second per user. Those are Nvidia's numbers about Nvidia's older hardware, and they should be read as vendor modelling, not as a benchmark.

The one third-party number

The figure with an outside name on it: Artificial Analysis measured 3,400 output tokens per second on the open-weight Gemma 4 31B at a 100,000-token context, which Nvidia's newsroom translates into "4x faster responsiveness" than the nearest alternative platform. Both the newsroom post and SiliconANGLE report it identically.

That is one model at one context length, and the comparison target is not named. It is still the most useful number in the announcement, because throughput-per-megawatt claims do not tell a developer whether their agent will feel quick.

What is actually shipping, and to whom

Nvidia names Nebius as the first production customer, deploying through its Token Factory inference platform, and says Groq itself plans to be an early adopter. SiliconANGLE, announcing from Hot Chips 2026, adds the backstory the Nvidia posts omit entirely: Nvidia acquired Groq for $20 billion in December 2025, licensing the technology and hiring founder Jonathan Ross and president Sunny Madra. That price appears in SiliconANGLE alone among these sources.

One naming wrinkle worth flagging: the developer blog calls the parts "Groq 3 LPU accelerators", SiliconANGLE calls them "LP30". The sources have not settled on a name for the thing.

What this means for developers

Nothing here is a chip you buy. It is capacity you rent, and the near-term question is whether Nebius, CoreWeave-class providers, or your existing inference vendor expose it - and at what price per million output tokens. Ask that before rewriting anything: a 4x responsiveness claim only matters if it survives contact with a price list.

The transferable idea is prefill/decode disaggregation. If Nvidia is building silicon around the premise that context processing and token generation are different workloads, that is a strong signal about how serving stacks are going to be shaped, and it is already visible in open-source serving today. Anyone running their own inference should be measuring the two phases separately - time-to-first-token against inter-token latency - because a single averaged number hides exactly the problem this hardware exists to solve.

Treat agent latency as a budget rather than a hope. The reason Nvidia can sell a dedicated decode chip is that agent loops emit tokens constantly, and every tool call restarts the cycle. If your agent makes ten sequential model calls, inter-token latency is multiplied ten times before a user sees an answer. That arithmetic is worth doing on your own traces this week; it costs nothing and it tells you whether specialised inference hardware is a line item in your future or an interesting headline.