Nvidia Turns Its Groq Deal Into Real Hardware
Nvidia’s enormous wager on Groq is leaving the PowerPoint stage and entering the data center.
On August 24, Nvidia announced that its Groq 3 LPX rack-scale inference system had entered full production. Cloud infrastructure provider Nebius will become the first AI cloud company to adopt it, with deployment expected later in 2026.
That timetable matters. Nvidia reached its approximately $20 billion arrangement with Groq in December 2025. Less than a year later, the resulting hardware is moving toward commercial operation. In semiconductor time, where new architectures can spend years wandering through laboratories, that is practically a sprint.
The company designed Groq 3 LPX to work alongside its Vera CPUs and Rubin GPUs within the broader Vera Rubin platform. Its job sounds simple: generate AI output extremely quickly. Yet that task has become one of the industry’s biggest infrastructure challenges.
According to CNBC, Nvidia senior director Dion Harris said Nebius expects to bring the equipment online before the end of the year.
The announcement marks more than another hardware launch. It shows how Nvidia plans to defend its position as AI moves beyond training gigantic models and toward running them constantly.
Training built the first phase of the AI boom. Inference may determine who wins the next one.
A $20 Billion Deal With an Important Asterisk
The Nvidia-Groq transaction deserves a little clarification because the word “acquisition” can make the story sound simpler than it is.
CNBC described the December transaction as a roughly $20 billion purchase of Groq assets, making it Nvidia’s largest deal on record. However, Groq characterized the arrangement as a non-exclusive licensing agreement, not a complete corporate takeover.
Groq continued operating as an independent company. Meanwhile, several senior Groq figures—including founder and former CEO Jonathan Ross—joined Nvidia. Nvidia also gained access to technology that could strengthen its inference business.
So, Nvidia did not simply swallow Groq whole and replace the sign on the office door. It secured valuable intellectual property, assets and talent while Groq remained active under new leadership.
That structure gave Nvidia something it urgently wanted: a specialized processor architecture built for fast and predictable AI inference.
It also gave the company a quicker path into an increasingly competitive corner of the chip market. Nvidia dominates AI training with its GPUs, but inference presents a different set of technical demands. Speed per user, power efficiency and token economics matter enormously.
Groq had spent years tuning its Language Processing Units, or LPUs, for precisely those demands.
Eight months after the deal, Nvidia now has a production-ready system bearing both names. That is a remarkably fast courtship by semiconductor standards. No awkward dinner phase—straight to a liquid-cooled rack.
Why Inference Has Become the Main Event
Training creates an AI model. Inference puts that model to work.
Every time a chatbot answers a question, a coding assistant produces software or an AI agent searches a database, the system performs inference. The hardware must process the prompt, evaluate the model and generate output one token at a time.
A token may represent a word, part of a word or punctuation. Generating one token does not sound demanding. Generating billions of them for millions of users, with almost no noticeable delay, is another matter entirely.
AI agents make the challenge harder.
A traditional chatbot might answer one question and wait. An agent can inspect files, write code, call external tools, check results, revise its plan and communicate with other agents. Each step can add context and trigger another round of inference.
Small delays quickly accumulate.
A task that requires 50 model interactions does not feel fast if each interaction introduces several seconds of waiting. The agent may be intelligent, but it moves like it is standing in line at a government office.
Nvidia therefore treats decode latency—the delay involved in producing tokens—as a major bottleneck. Groq 3 LPX attacks that specific problem.
As Blockchain.News explains, Nvidia sees low-latency generation as essential for responsive coding systems, multi-agent workflows and other real-time applications.
Inside the Groq 3 LPX Rack
Groq 3 LPX is not one unusually powerful chip sitting proudly on a circuit board. It is a complete rack-scale system.
Each rack contains 256 Groq 3 LP30 accelerators. Together, they provide 315 petaflops of FP8 inference compute, 128GB of on-chip SRAM and 40 petabytes per second of aggregate SRAM bandwidth. The chips communicate through direct connections that deliver 640 terabytes per second of scale-up bandwidth.
Those are industrial-strength numbers. Still, the most important ingredient may be the SRAM.
Many accelerators depend heavily on high-bandwidth memory located outside the processor. Groq’s architecture keeps fast SRAM directly on the chips. That design reduces the time spent moving information between compute units and memory.
The trade-off is capacity. SRAM is fast, but it is expensive and occupies considerable chip space. A single LPU cannot hold a large modern model by itself. Groq solves that problem by linking hundreds of accelerators and coordinating them as one system.
The rack also uses deterministic, compiler-scheduled execution. Instead of constantly deciding how to schedule work while a model runs, the system organizes much of that work beforehand. This approach can produce stable, predictable latency—especially useful when output must appear immediately.
According to Blockchain.News’ production overview, the architecture targets workloads where delays become visible to users, including coding, real-time decision-making and complicated agent loops.
The 3,400-Tokens-Per-Second Headline
Nvidia’s attention-grabbing performance claim comes from a benchmark conducted by Artificial Analysis.
Groq 3 LPX reportedly generated approximately 3,400 output tokens per second while running Gemma 4 31B with a 100,000-token input context. Nvidia said that result was four times faster than the nearest alternative platform tested and the fastest recorded performance for that model.
A more precise result reported by Blockchain.News was 3,431 tokens per second.
That is blisteringly quick. At 3,400 tokens per second, the system could theoretically produce 5,000 tokens in roughly 1.5 seconds. A platform operating at 100 tokens per second would need about 50 seconds to generate the same amount.
Groq 3 LPX also reportedly reached 3,382 tokens per second with a 10,000-token context. In Nvidia’s SPEED-Bench coding tests, it achieved a median rate of 4,767 output tokens per second, while 20% of tested tasks exceeded 5,500.
However, benchmarks always need context—quite literally in this case.
These numbers describe particular models, input lengths, configurations and tests. They do not mean every AI application will suddenly generate 3,400 tokens per second. Real performance varies with model architecture, request volume, software, networking, batching and other factors.
The result is meaningful. It just is not magic fairy dust for every workload.
Long Context Changes the Performance Equation
The 100,000-token input used in the headline benchmark is not a decorative detail.
AI systems increasingly need large context windows. A coding agent may examine an entire software repository. A research assistant could analyze dozens of documents. A customer-service agent might need years of account history plus the current conversation.
As that working context expands, the infrastructure must process more information before and during generation. Maintaining low latency becomes harder.
Long-running agents make the problem even messier. Every interaction can add new instructions, tool results, code, corrections and observations. The context grows while the agent works.
That explains why Nvidia emphasized performance at 100,000 tokens rather than showing only a short-prompt demonstration. The company wants Groq 3 LPX positioned as infrastructure for sustained, complex AI activity—not merely a chatbot firing off one-line answers.
Speed can also improve the agent’s reasoning process indirectly.
If token generation becomes faster, an agent can afford to perform more steps within the same period. It can inspect additional files, test more alternatives and verify its output before responding. That does not automatically make the underlying model smarter, but it gives the system more room to work.
A faster engine cannot improve the driver’s judgment. It can, however, let the driver complete more laps.
GPUs and LPUs Divide the Work
Nvidia is not replacing its GPUs with Groq LPUs. It is combining specialized processors.
The Vera Rubin NVL72 platform handles broad training and inference workloads. Groq 3 LPX extends that platform by concentrating on fast token generation. The two systems can work together rather than compete for the same role.
Rubin GPUs are well suited to computationally intensive and flexible operations, including processing large prompts. LPUs specialize in predictable, low-latency generation. Nvidia calls this approach “extreme codesign,” meaning it develops compute, memory, networking and software as parts of one coordinated platform.
This heterogeneous design reflects a larger change in AI infrastructure.
One processor type no longer needs to handle every stage of a workload. CPUs can orchestrate tasks. GPUs can process models and enormous contexts. LPUs can accelerate generation. Networking hardware moves data between them.
That specialization may improve speed and efficiency, but it also increases architectural complexity. Data centers must coordinate different processors without letting communication overhead erase their advantages.
Nvidia believes its networking technologies and integrated software stack can manage that complexity. If it succeeds, customers may view the company less as a GPU supplier and more as the architect of the entire AI factory.
That is a much larger—and potentially more profitable—role.
Nebius Gets the First Ticket
Nebius will be the first AI cloud provider to deploy Groq 3 LPX.
The company plans to install the accelerator alongside Vera Rubin NVL72 systems in its Nebius Token Factory. Developers will access the faster inference capacity through the same API they already use, reducing the need to migrate applications to an unfamiliar software stack.
That detail could prove crucial.
Spectacular hardware does not automatically attract customers if adopting it requires developers to rebuild their applications. Compatibility and easy access often matter nearly as much as raw speed.
Nebius gives Nvidia a commercial proving ground. Once the systems come online, customers can evaluate how LPX performs under real production traffic rather than carefully controlled benchmark conditions.
Groq also plans to deploy Groq 3 LPX alongside Vera Rubin in its own inference cloud. That arrangement creates an unusual loop: Groq’s technology entered Nvidia’s platform, and Groq will now become a customer for the Nvidia-branded hardware derived from it.
Meanwhile, CoreWeave is using Nvidia’s Spectrum-X Multiplane networking with Vera Rubin infrastructure. These partnerships show Nvidia assembling an ecosystem around its next-generation platform rather than launching a lonely rack and hoping someone finds an extension cord.
The real test will come when paying customers run unpredictable workloads at scale.
The New Battle Is Cost per Token
AI infrastructure buyers care about speed, but they also care about money. Lots of it.
Inference costs continue for as long as people use a model. A company may train a model periodically, yet serve billions of requests every month. Even small efficiency gains can produce significant savings at that scale.
Agentic AI intensifies the pressure. Agents may generate hundreds of thousands of tokens while solving one complicated task. Multiple agents can also collaborate, multiplying the amount of computation involved.
Fast token generation can improve user experience. Efficient generation can protect margins.
Nvidia argues that pairing Groq 3 LPX with Vera Rubin will help cloud providers increase infrastructure utilization while lowering cost per token. The company has not provided enough public, independently verified production data to settle the economics, however.
Power consumption, rack pricing, maintenance, software licensing and utilization rates will all affect the final calculation. A machine capable of astonishing peak speed may still disappoint financially if customers cannot keep it busy.
Still, the strategic direction looks clear. Nvidia wants to sell infrastructure for every stage of the AI lifecycle—from training and reasoning to the moment an agent begins pouring tokens onto a screen.
The GPU made Nvidia central to AI development. Token economics could help keep it there.
Nvidia’s Competitive Moat Gets Wider
Nvidia faces growing competition in inference.
Cloud companies are developing custom accelerators. Startups are building specialized processors. Customers are hunting for hardware that offers lower latency, better energy efficiency or freedom from Nvidia’s ecosystem.
The Groq deal gives Nvidia another weapon.
Instead of relying exclusively on general-purpose GPUs, the company can offer specialized LPUs inside the same platform. Customers gain access to a broader selection of compute without necessarily leaving Nvidia’s software and networking environment.
That makes the company harder to displace.
A rival can build a faster chip. Competing with Nvidia’s CPUs, GPUs, LPUs, networking products, software tools and cloud partnerships all at once is considerably more difficult.
Groq 3 LPX may also help Nvidia defend the lucrative inference market before specialized competitors gain too much ground. The company did not invent the LPU architecture internally, but it had the money and market power to bring Groq’s work into its own machine.
Twenty billion dollars is a spectacularly expensive shortcut. Nvidia appears determined to make it count.
Benchmarks Meet Reality Later This Year
Groq 3 LPX has reached full production, but the most revealing phase is still ahead.
Nebius expects to bring the racks online later in 2026. Once that happens, developers can test latency, throughput, reliability and cost under genuine production conditions.
Several questions remain.
Can the system maintain its responsiveness while serving many users simultaneously? How efficiently will it handle different model sizes? Will developers see the same benefits outside the benchmarked Gemma workload? And will lower latency justify the financial and operational cost of adding another processor class?
Nvidia’s published results give Groq 3 LPX an impressive opening act. They do not provide the final verdict.
Even so, the announcement captures where AI infrastructure is heading. The industry no longer measures progress only by how quickly companies can train a bigger model. It increasingly asks how quickly, reliably and cheaply that model can work.
Nvidia built its empire by supplying the machinery that trained the AI revolution. With Groq 3 LPX, it wants to supply the machinery that makes AI agents feel instantaneous.
The racks are rolling toward data centers. The tokens may soon follow at several thousand per second.
Now comes the fun part: seeing whether real-world AI can keep up with its new engine.
Sources
- CNBC — Nvidia says Groq racks will be online this year following $20 billion deal
- Blockchain.News — Nvidia Groq 3 LPX achieves full production for agentic AI
- Blockchain.News — Nvidia Groq 3 LPX hits full production and boosts inference speed
- Blockchain.News — Nvidia Groq 3 LPX achieves 3,431 TPS benchmark on Vera Rubin
The Kingy Brief
Follow The Kingy Brief.
One consequential launch, one pricing, limit, or shutdown change, one hands-on test, one exact prompt or Test Pack, and one try / watch / skip verdict.
Free · Choose your subjects · Double opt-in · Unsubscribe anytime