We followed the small coding pilot with a longer requirements-memory test on October 12, 2026. The 5 scorable full-history answers reproduced 240/240 final configuration facts; 1 full-history answer was unscorable. All 6 scorable handoff answers reproduced 288/288 facts after a fresh conversation received only a model-written note. These are repeated checks within two variants of one fictional workflow, not independent experiments for every fact.
Each scenario began with five campaign-routing profiles and 48 checked configuration facts. Thirteen accepted changes introduced partial reversions, a profile rename, retirement of the old alias, pause and fallback rules, replacement exclusions and a later correction. Three long blocks of archived records supplied conflicting old values. The records were explicitly marked as historical, so the task was to retain the current requirements rather than follow the most repeated number.
| Model and scenario | Full-history facts | Handoff facts | Full-history decisions | Handoff decisions | Strict JSON audits | Token estimate, US$ |
|---|---|---|---|---|---|---|
| Muse Wonder 1.0, scenario 1 | 48/48 | 48/48 | 10/10 | 10/10 | 4/4 | $0.2723 |
| Muse Wonder 1.0, scenario 2 | 48/48 | 48/48 | 10/10 | 10/10 | 4/4 | $0.2859 |
| GPT-6.1 Sol, scenario 1 | 48/48 | 48/48 | 10/10 | 10/10 | 4/4 | $0.1114 |
| GPT-6.1 Sol, scenario 2 | 48/48 | 48/48 | 10/10 | 10/10 | 4/4 | $0.1129 |
| Claude Sonnet 5.5, scenario 1 | 48/48 | 48/48 | 9/10 | 9/10 | 3/4 | $0.1729 |
| Claude Sonnet 5.5, scenario 2 | Unscorable | 48/48 | Unscorable | 10/10 | 0/4 | $0.2127 |
Before the final answer was generated, we saved the source conversation. One branch reconstructed the configuration from that full history. A separate branch asked the same model to write a memory note, then gave only that note and the original output/decision contract to a fresh API conversation. The note writer never saw the full-history final answer. The two branches answered identical final questions.
The note instruction allowed up to 700 words and permitted compact JSON. That is not a strict information-capacity limit: a minified JSON object can count as one whitespace-delimited word. We retain the note text, character count and token usage. This tests whether a model can write and use a faithful handoff for this task, not whether a tiny prose summary can preserve an arbitrarily large project.
Among scorable answers, the final full-history branches answered 49/50 proposal decisions correctly; the handoff branches answered 59/60. Across all 24 audits, 19 were valid strict JSON responses. An unscorable answer remains a failed response in the table; it is not counted as either correct remembered facts or proof that those facts were forgotten. The table keeps these measures separate so correct-looking remembered data cannot hide an invalid API response or a wrong operational decision.
Claude scenario one's full-history answer rejected the valid willow-forward proposal. Under the specified rule, paused Willow routes once to active Birch, which satisfies every eligibility check. Its handoff answer made a different error: it rejected spruce-route, even though its saved configuration correctly showed Spruce unpaused and the proposal met every condition. Both answers reproduced all 48 configuration facts correctly. These are observable rule-application mismatches, not missing configuration fields.
After a Claude baseline answer returned malformed overall JSON and trailing prose, we separated response-format compliance from recoverable requirement content. This was a post-observation scoring change. The original strict grades remain in the evidence. The additional pass first tries the whole response as JSON; if that fails, it may extract exactly one valid JSON value for each of the four unique named output fields. It does not rewrite values, guess missing fields or send a repair prompt. A recovered fact score therefore does not make that response valid for direct programmatic use. We requested JSON in the prompt; we did not enable provider-specific structured-output enforcement.
Claude scenario two's full-history response contained two sets of JSON values. Under the recovery rule, duplicate fields are ambiguous, so we did not pick a preferred answer or assign it a memory score. Its raw response and the failed original grade are retained.
The follow-up used an estimated US$1.1680 in tokens. Including the first pilot, the total was US$2.0762. The combined conservative allowance, including a 35% reserve for fees and exchange-rate uncertainty and higher possible input costs, was US$2.9452. That stayed below the original shared US$5 ceiling. These are estimates from returned usage, not attributable invoices. No additional subscription, paid judge or generation retry was used.
We used the same three native model IDs at medium effort. Two numeric variants do not establish a broad ranking; repeated facts and decisions are correlated. Tokenization, caching and internal reasoning differ, and Claude replayed native signed-thinking content while GPT and Wonder replayed visible responses. The handoff branch used a normal fresh API conversation: it did not call Meta Active Memory, native compaction, Muse Code or Wonder Full. It also does not establish performance over hours of real software work.
Download the complete follow-up evidence ZIP, including the frozen scenarios, original and revised grades, requests and responses, model-written notes, exact wrong answers and the combined cost ledger. API credentials are excluded.