kingy tested
What happened
Symptom
Repeated required-tool requests reuse the prompt prefix, end with finish_reason=length and return argument strings that cannot be parsed as JSON.
Conditions
Qualified 2026-10-16 Pacific on llama.cpp 0.6.0 build 11429 (d81235049), Qwen3-4B Q4_K_M; exact artifact SHA-256 in the package.
Apple M4 Pro, 24 GiB unified memory, macOS 26.3; CPU only, two threads, one slot, 2048-token context; loopback/offline.
For each repair, the same agent completed two seven-trial sequences in fresh server processes on the same Mac. Unchanged passing case runs were reused; the evidence index identifies them.
Correction
For this reproduced request, set cache_prompt=false while keeping the prompt, tool definition, temperature, seed and 64-token cap unchanged. Validate the complete tool call before using it.
Result
In each of two fresh-process runs: 0/3 baseline task passes, 3/3 corrected passes and 0/1 reverted passes. All original task checks and scoped completion checks passed. The offline audit recomputed acceptance from raw responses and rejected six damaged corrected-response controls.
What did not work
An initial qualification fixture expected a literal final period in a synthetic confirmation. The model omitted it. Those failed checks are retained; revision 2 made the confirmation instruction explicit without changing the initial tool request or cache correction.
- Use fixtures/006-cache.json to inspect the exact request; confirm that the failing response actually reports reused prompt tokens.
- Set only cache_prompt=false. Require zero cached tokens, finish_reason=tool_calls and complete original arguments.
- Supply the owned synthetic tool result using the returned call ID, then complete the confirmation turn. No web search is performed by the reproduction.
- Measure the cost of re-evaluating the full prompt before applying this mitigation to a larger workload. Keep sufficient output budget and continue validating arguments.
Sources & limits
Repair 006: full current qualification report and limits
Record checked 2026-10-16.
Same-agent repeatability checks on one Mac; no independent operator, second machine or human acceptance test.
Fixed synthetic tasks selected after exploration. The repeat counts do not establish a general reliability rate or support for other models and builds.
Runtime binaries and model weights are excluded. Exploratory failures, the failed first qualification fixture and all final trial responses are retained.
Client demonstrations identify owned reference integrations and injected inputs explicitly; they are not claims of a repaired Open WebUI release or a general llama.cpp bug fix.
This is a narrow mitigation of cache-dependent truncation at a fixed cap, not a universal tool_choice guarantee or a proven cache implementation root cause.
A warm 256-token control returned complete tool arguments with extra generated content. A larger cap is also a tested alternative for this fixture; do not hide that counterevidence.
The original report used Qwen3-8B and described plain-text-only replies. This 4B analogue has a different observed failure. No current president or web-search accuracy is tested.