Recurrent Depth Is Real. Its Role in OpenAI’s Astra Is Still a Report

A decade-old architectural idea has become a plausible new route for scaling AI reasoning. The public record does not yet show how much OpenAI’s Astra uses it, or whether it explains the model’s reported leap.

Last updated: September 1, 2026 · Research cutoff: September 1, 2026

OpenAI’s next major model has acquired a technical backstory before it has even launched. On September 1, The Information reported that Astra uses a constrained form of “recurrent depth,” a design that passes information through the same Transformer layers more than once before producing the next token.

The technique is real and the Astra connection is plausible, but the public evidence does not establish that connection as fact.

OpenAI has confirmed Astra’s name, impending release and unusually strong cybersecurity results. It has said nothing publicly about recurrent depth. The architecture claim comes from one anonymously sourced report, with no published system card, technical report, code, patent or independent corroboration behind it.

That leaves two stories to separate. One concerns a serious line of research that could change how models spend inference compute. The other concerns an unreleased OpenAI system whose architecture remains undisclosed.

Short answer

A recurrent-depth Transformer reuses one layer or block of layers several times. Each pass updates the model’s hidden state before the model emits a token. This can give a relatively small set of parameters much greater effective computational depth.

The trade is straightforward: fewer unique parameters do not mean less work. Every pass consumes compute and adds serial latency. More passes can improve difficult reasoning tasks, but they can also plateau, destabilize the model or make an answer worse.

The strongest public demonstration is Huginn, a 3.5-billion-parameter recurrent model trained on roughly 800 billion tokens. Huginn improved when researchers gave it more recurrent steps, but its own authors described the run as a proof of concept. Follow-up studies have strengthened the case for the architecture while exposing its costs.

As for Astra, the careful formulation is narrow:

A credible but unconfirmed report says OpenAI has incorporated a limited form of recurrent depth into Astra. OpenAI has not disclosed the architecture, and no public evidence shows how important recurrence would be to the model’s capabilities.

What is confirmed, reported and unknown

Claim	Evidence status
Recurrent depth is a real model architecture	Confirmed in published research
The idea predates today’s reasoning models	Confirmed
OpenAI plans to release Astra soon	Confirmed by OpenAI
Astra meets OpenAI’s Critical cybersecurity threshold	OpenAI’s assessment
Astra is more token-efficient than GPT-5.6 Sol on disclosed cyber evaluations	OpenAI’s assessment; not independently reproduced
Astra uses recurrent depth	Reported by The Information; unconfirmed
OpenAI constrained the recurrence to preserve readable reasoning traces	Reported by the same source; unconfirmed
Recurrent depth is a major cause of Astra’s performance	Unknown
Recurrent activations form a hidden verbal chain of thought	Unproven; evidence is mixed
More recurrence always improves a model	Contradicted by experiments

This separation matters because compatible clues are being mistaken for confirmation. Astra’s reported token efficiency could fit a recurrent architecture. OpenAI’s emphasis on chain-of-thought monitoring could fit a system that mixes hidden recurrence with written reasoning. Neither fact identifies the architecture.

How recurrent depth works

A conventional Transformer sends its hidden representation through a series of different layers:

hₗ₊₁ = Fₗ(hₗ)

Every Fₗ normally has its own learned parameters. A 32-layer model contains 32 separately parameterized Transformer layers.

A depth-recurrent model instead reuses a transformation:

hₜ₊₁ = Rθ(hₜ, e(x))

Rθ is the shared recurrent block. hₜ is the evolving hidden state. e(x) is usually a fixed representation of the input, reintroduced so the recurrent process does not lose sight of the original prompt.

The high-level flow looks like this:

tokens → prelude → [shared Transformer core ↻ r passes] → coda → next token

The recurrence runs through model depth during one prediction. It is different from a traditional recurrent neural network, which carries state across positions in a sequence. It is also different from Transformer-XL, which carries memory between text segments.

Most important, recurrent depth is not the same thing as written chain of thought. A reasoning model may generate hundreds or thousands of intermediate tokens before giving an answer. Those tokens occupy context, can be inspected by a monitor and require a fresh forward pass for every generated token.

A recurrent-depth model can spend more computation inside a forward pass, updating a continuous hidden state before it commits to the next token. It may still produce a written chain of thought afterward. The two methods can coexist.

This also separates recurrent depth from model distillation. Distillation trains one model from signals produced by another. Recurrent depth changes how one model reuses its own layers.

An old idea at a new scale

The current language suggests a new breakthrough. The lineage reaches back years.

The 2018 Universal Transformer repeatedly applied a shared attention-and-transition block across depth. Its optional adaptive computation mechanism let different token positions halt after different numbers of revisions. The paper showed that a Transformer did not need a unique parameter set at every layer to perform useful iterative computation.

ALBERT later used cross-layer parameter sharing to reduce the storage cost of BERT-style models. ALBERT normally ran for a fixed number of layers, so it was not an adaptive latent-reasoning system. It still demonstrated that extensive weight sharing could work in a successful language model.

Deep Equilibrium Models explored a related limit. Rather than unrolling a shared block for a chosen number of passes, a DEQ solves for a fixed point, treating the network as effectively infinite in depth. This can reduce activation memory, but it discards the path through the intermediate states. Modern recurrent-depth work uses a finite sequence of updates and asks whether additional updates improve the result.

The architecture therefore combines several established ideas: shared weights, iterative refinement, optional adaptive compute and direct input recall. What changed in 2025 was the scale of the demonstration.

What Huginn proved

The central modern paper is Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach, published at NeurIPS 2025. The researchers trained a 3.5-billion-parameter model called Huginn on 795 billion tokens using 4,096 AMD MI250X GPUs on the Frontier supercomputer.

Huginn contains a two-layer prelude, a four-layer recurrent core and a two-layer coda. The recurrent core was applied a randomly sampled number of times during training, averaging about 32 passes. The input representation was injected during each pass, and training backpropagated through only the final eight recurrences to keep memory manageable.

At 32 passes, Huginn still has eight unique Transformer layers. It executes 132 layer applications:

2 prelude layers + (4 recurrent layers × 32 passes) + 2 coda layers = 132

That distinction between unique layers and executed layers is the key to recurrent depth. The stored model remains 3.5B parameters. The computation does not remain small.

The benchmark pattern was encouraging. On ARC-Challenge, Huginn rose from 27.99 at four recurrences to 38.23 at 32. HellaSwag rose from 43.46 to 65.21. MMLU moved from 23.39 to 31.38. Easier tasks tended to saturate earlier; math and code often benefited from more passes.

The paper also included a matched 180-billion-token comparison. The recurrent checkpoint at 32 passes beat its one-pass counterpart on HellaSwag, 48.80 to 37.34, and on GSM8K with explicit chain of thought, roughly 9–10 percent to roughly 2 percent depending on the scoring rule. That is direct evidence that repeated hidden computation contributed something beyond sharing parameters.

The result came with unusually useful warnings. Two earlier large training attempts failed. One suffered representation collapse. Another learned to ignore the recurrent state, producing the same validation performance whether the core ran once or 32 times. The successful model required changes to normalization, initialization and learning rate.

Huginn also consumed far more compute than its parameter count suggests. The authors wrote that its pretraining FLOPs were close to those of a conventional 32B model, and that its heaviest test-time settings reached FLOP budgets comparable to a 50B fixed-depth Transformer. That is the source of claims that a 3.5B model can act like a 50B model.

The paper does not establish 50B-model capability. It establishes 50B-model computational load at the cited setting. On the authors’ broader comparisons, Huginn was roughly competitive with the first OLMo-7B generation but behind later OLMo models trained on larger, better-curated datasets. It beat general-purpose peers on some coding results while trailing the dedicated StarCoder2 models.

Huginn proved that recurrent-depth language-model pretraining can work at billion-parameter scale and convert additional inference FLOPs into better results. It did not settle whether recurrence beats unique layers when training data, total compute and serving latency are matched.

The evidence after Huginn

Later studies make the picture stronger and less magical.

Ouro, a family of 1.4B and 2.6B looped models led by ByteDance Seed researchers, was trained on 7.7 trillion tokens. Ouro introduced learned depth allocation and reported results competitive with larger models on reasoning-oriented tasks.

Its training history also exposed a boundary. The team initially explored eight loops, encountered instability and used four for the main models. Running beyond the trained depth often hurt. For Ouro-1.4B, MMLU fell from 67.45 at four loops to 64.49 at eight; ARC-Challenge also declined.

A 2026 preprint, How Much Is One Recurrence Worth?, offers one of the cleanest efficiency audits so far. Across 116 smaller pretraining runs, the researchers found that repeated layers carried less value than fresh layers. In one comparison, a 410M model looped four times matched the validation loss of a 580M non-looped model but consumed training compute closer to a 1B model.

That result is provisional and comes from smaller models. Its central warning is hard to dismiss: recurrent depth may save parameter memory while spending more FLOPs for the same loss.

Hidden computation is not a hidden monologue

The phrase “latent reasoning” invites a neat mental picture. The model silently works through intermediate steps, reaches a conclusion and speaks only when ready.

Benchmark improvement supports the presence of useful hidden computation. It does not reveal the shape of that computation. The recurrent state could be refining features, correcting errors, propagating constraints, retrieving representations or executing a distributed process with no clean sequence of intermediate propositions.

The strongest skeptical follow-up, Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer, probed Huginn’s recurrent states on arithmetic. The researchers found little consistent evidence that recognizable intermediate calculations emerged in order. With explicit chain of thought, Huginn scored far better than it did when that written reasoning was suppressed. Increasing the hidden recurrence from four to 32 produced only a modest gain in the no-chain-of-thought condition before performance plateaued.

Those probes have limits. A distributed nonlinear representation may not be recoverable through a simple lens. The study does not rule out latent reasoning. It does rule out treating “hidden chain of thought” as an observed mechanism.

“Hidden iterative computation” is the safer phrase.

Kingy.ai’s earlier guide to test-time compute describes the broader shift behind this work. Models can spend more inference compute by generating longer reasoning traces, sampling and ranking several answers, searching with tools or iterating inside latent space. Recurrent depth is one route through that design space.

What OpenAI has said about Astra

OpenAI has disclosed enough about Astra to make the architecture rumor consequential.

In its September 1 Path to Astra announcement, the company said Astra meets the Critical cybersecurity threshold under its Preparedness Framework. OpenAI defines that level to include autonomous discovery and exploitation of previously unknown vulnerabilities across hardened systems. It is the first OpenAI model assigned that designation.

OpenAI plans to make Astra available “soon,” while limiting its most advanced cyber functions to a smaller group of testers before expanding defensive access through Daybreak Blue.

The disclosed capability numbers are vendor-reported. OpenAI says Astra scored 100 percent on ExploitBench, then performed far better than GPT-5.6 Sol on an internal set of 20 recently disclosed V8 vulnerabilities while using fewer output tokens. During that evaluation, OpenAI says the model found two previously unknown vulnerabilities and used them in an exploit chain. Astra also reportedly assembled browser-compromise and operating-system privilege-escalation chains in expert-led tests.

These results used Daybreak Blue access rather than Astra’s default production configuration. No independent evaluator has reproduced the full set, and the internal benchmark is not public.

OpenAI also says it paused some Astra training after the Hugging Face incident involving a separate unreleased model. The company later restarted a large reinforcement-learning run under stricter security requirements. Astra itself was not involved in that incident.

The public announcements discuss alignment training, activation classifiers, system-level safeguards and additional chain-of-thought monitoring. They do not mention recurrent depth, looped Transformers or latent reasoning. OpenAI says it will publish more information in Astra’s system card at launch.

OpenAI’s Astra should not be confused with Google DeepMind’s Project Astra, an unrelated multimodal-assistant research project.

The recurrent-depth report

On September 1, The Information reported, citing one anonymous source, that Astra uses constrained recurrent depth. OpenAI has not confirmed it.

The report deserves attention. The Information has a record of reporting internal model developments, and the described technique fits a live research direction. Technical plausibility still falls short of verification.

No named OpenAI employee, system card, technical paper, code release or patent has tied Astra to recurrent depth. Other reports about Astra’s cyber announcement have not independently sourced the architecture claim. Social posts and discussion threads trace back to the same article.

The evidence supports careful attribution, not a declaration that recurrent depth is Astra’s secret.

It supports even less certainty about causation. Astra’s results could reflect pretraining scale, data, reinforcement learning, tool use, long-horizon agent training, inference orchestration, model routing or several architectural changes. A recurrent component, if present, could be central, limited to selected tokens or tasks, or experimental rather than foundational.

Why chain-of-thought monitoring matters

Recurrent depth creates a real safety question. Reasoning models often externalize some useful computation in text. Automated monitors can inspect that text for suspicious plans, unauthorized actions or attempts to bypass controls.

A 2025 multi-organization position paper, Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety, argues that these traces provide an imperfect but valuable monitoring surface. Kingy.ai previously examined the paper in its chain-of-thought monitorability summary.

Moving more computation into recurrent activations could reduce how much useful reasoning appears in readable text. That would weaken one safety signal. It would not transform a transparent machine into an opaque one; neural-network activations were already difficult to interpret, and written reasoning is incomplete and sometimes unfaithful.

OpenAI’s public Astra plan suggests a layered approach. The company says it will monitor both reasoning and actions, use classifiers to detect unauthorized behavior and stop a task when necessary. What it has not explained is whether the model contains recurrent states, how those states would be monitored or whether its written reasoning remains causally important to difficult decisions.

There is a second distinction for readers. A user-facing summary of reasoning is not necessarily the raw chain of thought used by a safety monitor. OpenAI has previously described showing model-generated summaries rather than raw reasoning traces. Public transparency and internal monitorability are separate questions.

The economics are more complicated than “small model, big brain”

Recurrent depth could improve model economics in specific settings.

Reusing weights reduces the amount of model data that must be stored, moved between chips or synchronized during training. A smaller weight footprint may fit more readily on limited hardware. The model can also vary its depth, exiting early on easy tokens and spending more passes on difficult ones.

Production systems still have to pay for the passes. Each loop runs attention and feed-forward operations again. The loops are sequential, so adding them can increase latency even when the weights stay resident in fast memory. Dynamic per-token exits complicate batching because different tokens may require different amounts of work. Recurrent key-value caching also demands specialized inference code.

Output-token efficiency should not be confused with total efficiency. A model that produces a short answer after 32 hidden passes may use fewer visible tokens while consuming more GPU time than a model that writes a longer rationale with fewer internal passes.

A commercially meaningful evaluation must ask whether a recurrent model delivers better accuracy, latency and cost at the same quality target under production conditions.

No public Astra data answers that question.

What Astra’s system card needs to disclose

If OpenAI confirms recurrent depth, the name alone will tell us little. The useful disclosure would answer concrete questions:

1. Which part of Astra is recurrent?
2. How many times can the shared block run?
3. Is recurrence fixed, randomly sampled or selected dynamically by token or task?
4. Was it used during pretraining, reinforcement learning, inference or all three?
5. How does Astra compare with a non-recurrent baseline at equal training and inference FLOPs?
6. Do more loops ever degrade performance?
7. How much latency and total compute does hidden recurrence add?
8. How are recurrent activations evaluated for dangerous planning or unauthorized intent?
9. How much of Astra’s difficult reasoning still depends on explicit chain of thought?
10. Are the disclosed cyber results representative of the production model or the Daybreak Blue configuration only?

Until those answers exist, claims about recurrent depth’s contribution to Astra remain interpretation.

Verdict

Recurrent depth deserves serious coverage. Universal Transformers established the basic pattern. Huginn showed that a billion-parameter language model could use extra recurrent passes to improve difficult tasks. Ouro and later scaling work suggest the idea can survive larger datasets, while showing that useful depth is bounded and compute costs remain substantial.

The research supports three conclusions. Shared layers can perform useful iterative computation. Parameter savings do not erase FLOP and latency costs. Better answers after more passes do not prove that a readable chain of hidden thoughts exists.

The Astra claim sits on a different evidentiary level. One credible report says OpenAI uses a constrained version of the technique. OpenAI has confirmed the model, its Critical cyber designation and added chain-of-thought monitoring, but not its architecture.

The next reliable evidence should come from Astra’s system card. If that document does not describe the model’s architecture, recurrent depth may remain a well-sourced rumor attached to a technically plausible story.

Frequently asked questions

What is recurrent depth in an AI model?

Recurrent depth is an architecture in which the same Transformer layer or block is applied repeatedly to an evolving hidden state. This increases the number of computational steps without adding a new set of parameters for every step.

Is recurrent depth the same as chain of thought?

No. Chain of thought usually refers to intermediate reasoning expressed as tokens. Recurrent depth performs repeated computation inside the model’s latent activations. A model can use both methods.

Does OpenAI’s Astra use recurrent depth?

The Information reported that it does, citing one anonymous source. OpenAI had not confirmed the architecture as of the September 1, 2026 research cutoff.

Does recurrent depth make inference cheaper?

It can reduce parameter storage and weight-transfer costs, but every recurrence consumes compute. Total cost depends on the loop count, hardware, serving implementation, output length and quality target.

Is a 3.5B recurrent model equivalent to a 50B model?

No. Huginn’s paper said its highest test-time settings used a FLOP budget comparable to a conventional 50B model. That was a compute comparison, not a claim of equal parameters or general capability.

Can a model keep improving if it runs more loops?

Not reliably. Huginn’s gains saturated at different depths by task. Ouro and earlier recurrent-network research found that excessive iteration can reduce accuracy or destabilize representations.

Sources and methodology

This article separates peer-reviewed research, preprints, OpenAI’s first-party statements and secondary reporting. Benchmark figures attributed to OpenAI are presented as vendor-reported because the relevant internal evaluations and production configuration are not independently reproducible.

The Astra architecture claim is attributed only to The Information. Search results, derivative reporting and social posts were reviewed for independent corroboration; none available by the cutoff supplied a separate architectural source. Absence of public confirmation does not prove the report is wrong.

Core sources:

• OpenAI, Path to Astra: Critical Capabilities and Frontier Safeguards, September 1, 2026.
• OpenAI, Responding to the Next Frontier of Critical Cyber Capabilities, August 7, 2026.
• OpenAI, Pacing Model Development in Response to Emerging Cyber Capabilities, August 18, 2026.
• The Information, OpenAI Technique in ‘Astra’ Model Sparks Security Concerns, September 1, 2026.
• Dehghani et al., Universal Transformers, ICLR 2019.
• Bai, Kolter and Koltun, Deep Equilibrium Models, NeurIPS 2019.
• Geiping et al., Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach, NeurIPS 2025; official code; Huginn model card.
• Lu et al., Latent Chain-of-Thought? Decoding the Depth-Recurrent Transformer, 2025 preprint.
• Zhu et al., Scaling Latent Reasoning via Looped Language Models, 2025 preprint, revised 2026.
• Schwethelm, Rückert and Kaissis, How Much Is One Recurrence Worth?, 2026 preprint.
• Korbak et al., Chain of Thought Monitorability: A New and Fragile Opportunity for AI Safety, 2025.

Kingy.ai’s wider evidence and freshness conventions are described in the Kingy AI Reports methodology.