Python splitlines breaks streamed AI answers containing Unicode separators

A reference client split real model SSE data at U+0085, U+2028 and U+2029. Framing only ASCII CR/LF preserved the complete Unicode answer.

Qualified 2026-10-16 Pacific · Same-agent local repeatability

Qualified 2026-10-16 Pacific on llama.cpp 0.6.0 build 11429 (d81235049), Qwen3-4B Q4_K_M; exact artifact SHA-256 in the package.

Apple M4 Pro, 24 GiB unified memory, macOS 26.3; CPU only, two threads, one slot, 2048-token context; loopback/offline.

For each repair, the same agent completed two seven-trial sequences in fresh server processes on the same Mac. Unchanged passing case runs were reused; the evidence index identifies them.

Same-agent repeatability checks on one Mac; no independent operator, second machine or human acceptance test.

Download full evidence cohort

412,621 bytes

SHA-256: 8a3e0d0b0f217f30e583450c7073dd1558ce75b9bb8f0e3e8f348e6260a28b93

Observed symptom

A Python streaming client loses text or raises JSON/framing errors when str.splitlines treats Unicode characters inside a data payload as protocol line boundaries.

Correction

Decode UTF-8 incrementally and frame SSE lines only on CRLF, CR or LF. Preserve Unicode separators inside JSON data. Require a complete stop event and DONE before accepting the answer.

  1. Inspect the real retained SSE bodies and owned expected text in fixtures/008-sse.json. Confirm the actual raw Unicode separators are present.
  2. Replace the broad splitlines framing pattern with the bounded decode_sse reference in client_fixes.py or a maintained SSE client with equivalent framing behavior.
  3. Compare complete decoded output with the original expected string, including composed/decomposed accents, CJK, emoji and each separator.
  4. Replay the same real body in whole, one-byte and seven-byte chunks. Require identical complete text under every partition.
  5. Reject truncated UTF-8, incomplete streams and length termination. The reference decoder supports text-only chat responses and is not a browser EventSource replacement.

Results

In each of two fresh-process runs: 0/3 baseline task passes, 3/3 corrected passes and 0/1 reverted passes. All original task checks and scoped completion checks passed. The offline audit recomputed acceptance from raw responses and rejected six damaged corrected-response controls.

Failed attempts and controls

The first exploratory analysis script mishandled a null content delta and stopped. Its original wire body was retained and successfully analyzed by the corrected decoder; qualification uses the frozen decoder.

Limits

Same-agent repeatability checks on one Mac; no independent operator, second machine or human acceptance test.

Fixed synthetic tasks selected after exploration. The repeat counts do not establish a general reliability rate or support for other models and builds.

Runtime binaries and model weights are excluded. Exploratory failures, the failed first qualification fixture and all final trial responses are retained.

Client demonstrations identify owned reference integrations and injected inputs explicitly; they are not claims of a repaired Open WebUI release or a general llama.cpp bug fix.

The failing client is an owned reference integration using the documented problematic framing pattern. No installed Open WebUI release or browser display was tested.

The live tests cover one fixed mixed-Unicode response on the pinned local model. Prepared FF-06 fixtures remain separate software controls, not additional live-model trials.

This correction repairs transport framing; it cannot recover text a model never generated.

Evidence

The ZIP includes exact requests, raw responses, the indexed complete case runs, failed initial attempts, frozen runner/client code, the offline audit, all file checksums and the selection record. It contains no model weights or runtime binary.

Primary protocol or upstream reference. The current result above is limited to the identified artifacts and fixture.

Read this repair in Fix Finder →