kingy tested
Tested repair · llama.cpp + Python client · llama.cpp 0.6.0 b11429; Qwen3-4B Q4_K_M; Python 3.14.0
Check this fix
- Applies when
- json.loads on the HTTP body raises JSONDecodeError at data: even though the local server returned HTTP 200.
- Change to inspect
- For a client that expects one complete Chat Completion JSON body, set stream=false. Keep the endpoint, task, response schema and other generation settings unchanged.
- Expected result
- 0/3 baseline full-task passes, 3/3 corrected passes and 0/1 reverted passes. Each repair uses its own seven attempts; corrected generations are not counted twice.
- If it still fails
- Keep the original failed output and check the conditions and verification steps below. Retained failed attempt: All three baseline attempts and the reverted attempt failed the unchanged full-task checker; raw requests, responses and client errors are retained.
Tested or reviewed 2026-10-27. Verify the original task on your own version and settings; this case does not prove a universal fix.
Read the verification steps ↓ · Report your outcome ↓What happened
Symptom
json.loads on the HTTP body raises JSONDecodeError at data: even though the local server returned HTTP 200.
Exact environment and tested conditions
Same exact pinned local runtime/model and original synthetic task; see evidence packet for full requests.
Owned Python reference integration; POST /v1/chat/completions on a loopback-only server.
Correction
For a client that expects one complete Chat Completion JSON body, set stream=false. Keep the endpoint, task, response schema and other generation settings unchanged.
Result
0/3 baseline full-task passes, 3/3 corrected passes and 0/1 reverted passes. Each repair uses its own seven attempts; corrected generations are not counted twice.
What did not work
All three baseline attempts and the reverted attempt failed the unchanged full-task checker; raw requests, responses and client errors are retained.
- Check the request stream flag and response Content-Type before interpreting a JSONDecodeError as a model failure.
- If the application needs one complete JSON response, change only stream=true to stream=false. A streaming interface instead needs a maintained SSE decoder and delta aggregation.
- Parse the outer completion envelope, require one completed choice, then parse message.content and validate every original field and type.
- The retained baseline contains the actual SSE wire body; the corrected response contains the exact expected wrapped record.
Sources & limits
Four client/configuration repairs: full qualification report, exact changes and limits
Pinned llama.cpp server API documentation for tested build d81235049
Record checked 2026-10-27.
Qualified October 27, 2026 by the same agent in one fresh server process on one Apple M4 Pro Mac with 24 GiB unified memory; no independent operator, second machine or human usability outcome.
Three baseline failures, three corrected full-task passes and one reverted failure per repair. These are fixed synthetic integration tasks chosen after exploration, not a general reliability estimate or evidence of user demand.
llama.cpp 0.6.0 build 11429 commit d81235049, Qwen3-4B Q4_K_M, Python 3.14.0; CPU-only four threads, one slot, 2048-token context, reasoning off, temperature 0, seed 42 and cache_prompt=false.
The failing clients/settings are owned reference integrations. These tests do not establish an installed application defect or an upstream runtime fix. Runtime binaries and model weights are excluded from the package.
This correction changes the transport mode for an owned whole-body client. It does not repair an SSE parser or measure streaming latency, and is separate from the existing Unicode SSE framing case.