# Repaired candidate 1.5.4 paired PDF comparison protocol

This protocol is prepared before the repair agent or root opens the new question text or reference values. It defines planned measurements, not results. Repaired candidate 1.5.4 remains unqualified. Its consumed v1.4/v1.5 failure repairs and earlier development controls do not establish unseen accuracy.

Use the recorded Apple M4 Pro with 24 GiB unified memory and macOS 26.3. Record hardware, OS, AC power, power mode, free disk, memory pressure, swap and thermal state before and after every session. This is a shared Mac; do not claim it is isolated or terminate unrelated jobs. Only one comparison model runs at a time.

Both workflows use identical Qwen3-4B Q4_K_M bytes: 2,497,280,256 bytes, SHA256 `7485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5`. Kingy uses reviewed runner 1.5.4, llama.cpp b11429 and alias `kingy-qwen3-4b`. The native LM Studio GUI uses loaded identifier `qwen3-4b`. Generation seed is 271828 on every question and neutral warmup; temperature zero, reasoning off, context 8192, ten CPU threads and one parallel sequence. Record effective native settings, app/build, engine binaries, model bytes, official stock RAG revision/configuration, embedding artifact, retrieval limit, affinity threshold and native parser/chunker settings. Revalidate native artifacts and the previously measured effective settings with a fresh neutral warmup before method freeze. The new blank template must have the same preset and RAG configuration; fresh-corpus attachment hashes are checked before every submission. A declared setting or a settings-file hash alone is not proof of its effective value.

The native arm uses the unmodified official app document workflow and factual request from `native-user-template.txt`. That request and the format-only native capture adapter remain byte-identical to the v18 method. Use the consumed-data calibrated stock RAG configuration (eight retrieved chunks, affinity threshold 0.5) and record any actual mismatch before the new bank is exposed. Every native question is the first message of an empty chat containing all ten identical PDFs exactly once. Record original versions, EOS completion, native processed prompt, answer, actually emitted citations, source mappings, configuration and documented logger events. Do not add a new judge, retrieval mechanism, follow-up or tuned prompt after viewing outcomes.

The new independently authored pack is planned to contain ten distinct English text-bearing PDFs, 30 pages and 40 held-out questions. No source text, expected answer or reference support span is accessed while preparing this infrastructure. First freeze method, scorer, aliases, thresholds, paired-analysis rules, native UI helper, source files and settings receipts. Only then may the root run the schedule builder. That program retains ID/split metadata, hashes the PDF/question/reference bytes and produces three deterministic permutations with randomization seeds 271828 plus repetition. It does not parse reference values or print question text. The complete freeze identifies the schedule and every source/method/corpus artifact before the first held-out inference.

Predeclared session order is **A1, B1, B2, A2, A3, B3**, where A is Kingy and B is native LM Studio. Each arm has three fresh model sessions of 40 paired questions, with identical within-pair order. New measurement sessions use new output folders. The native coordinator may resume capture of an existing original pending attempt only after checking an identical plan and retained-record hashes; it must never resubmit that question. Warmup is one neutral request, `Reply only READY.`, with no system instruction or documents, temperature zero, seed 271828, top-p 1, cache disabled and reasoning off. Keep cold startup, corpus import/indexing, warmup and question timing distinct. Record cache reuse; do not claim OS caches were flushed.

Every planned question gets one original attempt. Retain errors, incomplete generations, timeouts, original raw output, all per-field Kingy phases, extra native model outputs, citation mapping ambiguity and unused planned IDs. A complete usable answer must finish within 90 seconds. Kingy records wall time around the complete `runner.ask` return and rejects an over-deadline result. Native timing runs from app submission through the unique documented final-answer event, including native preprocessing/retrieval/embedding/inference; title inference and export overhead are reported separately. Unavailable or incomparable timing stays unavailable. Additional API work is included in request counts.

Once held-out data is consumed, candidate code, prompts, settings, source selection, scorer, pass rules and scheduled order remain frozen. Do not retry individual failures, tune on the new bank, regenerate answers or quietly discard work. A transport/backend/GUI interruption that prevents completion may trigger a whole fresh replacement of that same session, using the frozen order/configuration, only when declared before reading quality grades or reference values. Use a new folder such as `B2-recovery1`. Preserve and later score every retained original partial attempt in an interrupted-session appendix. Do not splice the partial and replacement into a best-of result. If quality outputs were already reviewed, a replacement cannot qualify as an unseen continuation; disclose it separately. Coordinator interruptions do not establish competitor reliability.

Apply the separately frozen common scorer and paired-analysis specification. Every essential requested fact/qualifier and its linked source support must pass. Native citation markers and uniquely mapped quote pages are accepted under the frozen format adapter; automatically displayed source cards are not automatically credited. Report factual correctness, complete supported answers, emitted-citation support, explicit page formatting and unknown handling separately. Preserve unresolved cases in lower/upper bounds and denominators. Each complete run must independently satisfy the declared lower-bound thresholds: 90% complete supported-answer success, 95% emitted-citation support and 90% correct abstention.

Forty unique paired questions are the statistical unit; repetitions measure repeatability and are not 120 independent users or questions. Publish per-session denominators and paired descriptive/uncertainty results. Post-output human review must be labelled supplemental and not blinded if the reviewer knows the systems. The 1.4 scorer audit found native paraphrase false negatives; 1.5 uses reviewed consumed-data semantic aliases frozen before the new bank. This reduces a known grading issue but does not prove every unseen paraphrase will be classified correctly.

Kingy's client/server keep the existing deny-egress profile. Native LM Studio retains its vendor sandbox; its egress enforcement remains unverified because an outer Seatbelt profile conflicted with Chromium initialization. This study cannot establish comparative privacy. Claims are limited to these configured complete workflows, this synthetic corpus and this Mac. No general model-accuracy, speed, hardware-coverage, usability or cheapest-market claim follows from this comparison alone.

This study is comparison-v19. Candidate 1.5.4 repairs the consumed v18 failures through field-specific request isolation, explicit-document retention, per-field current/archive handling, lexical inspection/return families, and page-diverse bounded source candidates. Neither inference nor ranking uses grading references. The shared scorer remains version 1.4.0, including its conservative finite-composition limits. The independently checked native direct-menu transport is frozen before the corpus is created; its single-use preparation receipts never submit questions. Previous study originals, scores and public reports remain unchanged. A new independently authored corpus follows the strict SCORING.md contract and a separate source review. AI-agent reviews are not human review or independent hardware reproduction. Root does not read the new grading key before all six sessions finish. Active workflow 1.4 remains until every quality, package, authentication, offline and public verification gate passes.

Each Kingy field has an isolated retrieval, selection and sufficiency pipeline, with at most four model requests for a two-field question. The result preserves every field attempt and full question latency. A finite participant-role witness guard can decline unfamiliar synonyms and does not by itself prove entailment. The complete 90-second deadline and two-field bound are enforced before accepting the answer.