kingy tested
Tested repair · llama.cpp + Python client · llama.cpp 0.6.0 b11429; Qwen3-4B Q4_K_M; Python 3.14.0
Check this fix
- Applies when
- The response is valid JSON with id and text at the root, but the application requires those fields inside kingy_record.
- Change to inspect
- Replace plain response_format={"type":"json_object"} with the tested nested json_schema response format carrying the unchanged original schema. Keep the prompt and expected source facts unchanged.
- Expected result
- 0/3 baseline full-task passes, 3/3 corrected passes and 0/1 reverted passes. Each repair uses its own seven attempts; corrected generations are not counted twice.
- If it still fails
- Keep the original failed output and check the conditions and verification steps below. Retained failed attempt: All three baseline attempts and the reverted attempt failed the unchanged full-task checker; raw requests, responses and client errors are retained.
Tested or reviewed 2026-10-27. Verify the original task on your own version and settings; this case does not prove a universal fix.
Read the verification steps ↓ · Report your outcome ↓What happened
Symptom
The response is valid JSON with id and text at the root, but the application requires those fields inside kingy_record.
Exact environment and tested conditions
Same exact pinned local runtime/model and original synthetic task; see evidence packet for full requests.
Owned Python reference integration; POST /v1/chat/completions on a loopback-only server.
Correction
Replace plain response_format={"type":"json_object"} with the tested nested json_schema response format carrying the unchanged original schema. Keep the prompt and expected source facts unchanged.
Result
0/3 baseline full-task passes, 3/3 corrected passes and 0/1 reverted passes. Each repair uses its own seven attempts; corrected generations are not counted twice.
What did not work
All three baseline attempts and the reverted attempt failed the unchanged full-task checker; raw requests, responses and client errors are retained.
- Retain the original application schema separately from the prompt. Parsing JSON is only the first check.
- For the tested Chat endpoint use response_format type json_schema with json_schema.name, json_schema.strict and json_schema.schema; put the original schema in that schema field.
- Validate the exact original object shape, required fields, types and source facts after generation. The fixture requires kingy_record containing integer id 17 and text ALPHA.
- Do not widen the checker to accept an unwrapped baseline. An unchanged checker rejects all four baseline/reverted outputs in the packet.
Sources & limits
Four client/configuration repairs: full qualification report, exact changes and limits
Pinned llama.cpp server API documentation for tested build d81235049
Record checked 2026-10-27.
Qualified October 27, 2026 by the same agent in one fresh server process on one Apple M4 Pro Mac with 24 GiB unified memory; no independent operator, second machine or human usability outcome.
Three baseline failures, three corrected full-task passes and one reverted failure per repair. These are fixed synthetic integration tasks chosen after exploration, not a general reliability estimate or evidence of user demand.
llama.cpp 0.6.0 build 11429 commit d81235049, Qwen3-4B Q4_K_M, Python 3.14.0; CPU-only four threads, one slot, 2048-token context, reasoning off, temperature 0, seed 42 and cache_prompt=false.
The failing clients/settings are owned reference integrations. These tests do not establish an installed application defect or an upstream runtime fix. Runtime binaries and model weights are excluded from the package.
This tests absent schema enforcement in plain JSON mode. It does not claim full JSON Schema dialect support. The fixture explicitly asks for unwrapped source facts while the original application schema requires a wrapper; these deliberately different contracts make the boundary observable. Existing schema-envelope and unsupported-composition cases remain separate.