kingy tested
What happened
Symptom
HTTP 200 with nonempty reasoning_content, empty visible content and finish_reason length after eight completion tokens.
Conditions
Qualified October 15, 2026 on llama.cpp 0.6.0 build 11429 (d81235049384534c167caea52b85a694f6103d14), Qwen3-4B Q4_K_M.
macOS 26.3 build 25D125, Apple M4 Pro / Mac16,7, 24 GiB unified memory; CPU-only inference.
Separate-agent rerun from a fresh extraction and fresh server processes on the same physical Mac.
Correction
For the retained task, increase max_tokens from 8 to 1024 while preserving thinking, prompt and other settings. Check that the visible answer completes and the stop reason is stop.
Result
Author revision 2 and the separate-agent rerun each had 0/3 baseline passes, 3/3 corrected passes and 0/1 reverted passes. Every corrected answer was exactly 42, finished with stop and used 698 total completion tokens with zero cached tokens.
What did not work
The initial 512-token correction failed all three author checks: visible content remained empty and finish_reason remained length.
An earlier port-preflight abort occurred before a model request and is retained separately.
- Inspect both visible content and reasoning_content, total completion usage and finish_reason. HTTP 200 alone does not mean the task completed.
- Keep thinking enabled. The test holds temperature 0, seed 42, deepseek reasoning format and a 2,048-token context unchanged.
- Raise only max_tokens to 1024 for this task. The 27-token rendered input plus the 1,024-token reserve fit the tested context.
- Require exactly 42 as the visible answer, nonempty reasoning and finish_reason stop. Retain any cap exhaustion as a failure.
- Download the package for every failed and passing attempt, including the failed 512-token correction and the offline evidence audit.
Sources & limits
Repair 003: full qualification report and limits
Record checked 2026-10-15.
Same-host separate-agent reproduction, not independent human testing or a second-machine result.
These fixed synthetic tasks and repeated settings do not establish a general reliability rate, other-model behavior or minimum hardware requirements.
Original failures, failed correction proposals, prompts, outputs, settings and checksums remain in the evidence package. Model weights and runtime executables are not included.
Qualification ran October 15, 2026. Publication does not make this a test of later software or model revisions.
The precise reasoning-versus-answer token split is unknown. The cap was selected during qualification; it is not a held-out result or a universal minimum.