← Kingy Local AI

FULL THREE-PAIR DOCUMENT STUDY · WORKFLOW 1.5.5

Kingy and native LM Studio: three paired PDF repetitions

document-comparison-v20-20261018.1 · analysis completed 2026-10-18T05:25:49.143945+00:00

40 unique question clusters × 3 paired repetitions = 120 attempts per workflow.

The same five PDFs/30pages and40held-out questions were attempted in three original repetitions per workflow. Model bytes,37 source files, settings and common scorer were frozen before a separate agent created the new corpus. Executed native preparation transport changed during collection, as disclosed in the operational and independent eligibility reviews. Every original remains in its denominator. Candidate1.5.5 passed the frozen Kingy numeric targets; the separate release decision records all remaining launch gates.

Each pair contains the same 30 answerable and 10 unknown questions. Repetitions measure repeatability on the declared system; these are 40 unique cases, not 120 independent questions or an independent physical reproduction.

Scoring repair and independent checks

The new rubric requires every requested fact, condition and time anchor, even when the answer is copied from a real source. Correct concise values and explicit semicolon-separated fields can receive the same credit as longer answers. A separate agent reviewed the implementation with independent adversarial controls before this corpus was created. The linked receipts identify the exact sources, tests, failures and corrections. This is a bounded AI-assisted review, not human validation or proof that every possible paraphrase can be judged.

Independent engineering review and exact source pins · Independent every-item source and task review · Preserved v19 study and its scoring limitations

Inference repair and independent checks

Each requested field now has separate retrieval, evidence selection and sufficiency checks, within one 90-second question deadline. The shared task-completeness scorer is version 1.5.0, with contextual concise-answer checks, role-bound compound obligations and source-location support matching. Consumed-corpus regression and independent synthetic controls preceded this separately authored fresh corpus. Finite participant-role checks can decline unfamiliar wording; they do not guarantee truth.

Independent engineering review and exact source pins · Independent every-item source and task review · Candidate and scorer repairs with development evidence

Original completion and prospective method adherence

All 37 frozen source files were preserved. Executed native preparation transport was amended after collection began; preserved source hashes do not establish identical executed controls or strict prospective-method adherence. The seal’s legacy methodUnchanged field checks source hashes and is not a whole-execution adherence finding.

Retained inference evidence and input equivalence grant no automatic exemption from the prospective method or launch gates. Independent original-completion, deadline and settings eligibility · Independent post-start preparation transport assessment; strict method adherence separately adjudicated

Fresh package installation check

The actual fresh package check failed. 3 of 4 public selftests passed. Numeric results on the fresh comparison corpus do not override this package gate; candidate1.5.5 remains held.

Actual fresh package check exited1 at selftest assertion: selftest-cross desk_extension incorrectly answered with tire-check text. Original output retained; no retry or frozen candidate change.

Failed public selftest: selftest-cross. All four original outputs and the summary remain retained.

Authentication and declared client/server egress checks: passed. This separate result does not turn a failed answer selftest into a passed installation. Fresh ZIP extraction on recorded Mac with declared prerequisites; not novice-user research or separate-machine reproduction.

Fresh extraction/selftests/authentication/denied-egress check · All four actual public recipe selftest originals, including failures · Actual public selftest summary; no assumed pass · Actual recipe authentication and egress checks

Final independent source check and release decision

The independent strict prospective-method adherence gate did not pass; executed native preparation transport changed after collection began. Fresh package verification did not pass: 3/4 self-tests passed; authentication and denied-egress checks passed. Candidate1.5.5 is held; active workflow1.4 remains.

All240 originals are complete within the declared90-second answer boundary. The final AI-assisted audit reviewed all40 questions,41 source obligations,30 PDF pages and80 distinct response/source bundles; no demonstrated definite false pass or false definite failure was found under the frozen finite-recognition contract. Strict prospective transport/code-freeze adherence did not pass, and the fresh package selftest passed only3/4. Candidate1.5.5 remains held despite meeting the frozen numeric Kingy targets.

The bounded final scoring audit passed within its declared scope. No overall comparative quality winner is claimed. Unblinded AI-assisted every-case review on this same Mac, plus independent PDF extraction, original-character quote/offset/anchor containment and exact frozen-score replay. All fixed definite passes and definite supported-answer failures are substantiated; unresolved semantic/format outcomes remain nonpasses and are not upgraded or treated as proven factual errors. This is not human adjudication, population accuracy or an overall competitor ranking. The separate agent reviewed 80 distinct response bundles covering all 240 originals. This is an unblinded AI-assisted review, not human adjudication. The original scores and responses are preserved.

Independent final scoring and source audit · Every independently reviewed answer/source bundle · All failed or unresolved Kingy originals · Actual launch decision and all gates

Observed differences in both directions

These are observations on this fixed corpus and configuration, not a general product ranking.

Complete paired accounting

All A1/B1, A2/B2 and A3/B3 attempts are retained. Confirmed passes form the conservative rate. The upper bound additionally includes unresolved judgments. These bounds are not confidence intervals. Canonical facts and complete supported answers are reported separately. Displayed native retrieval cards count only when claimed in the answer; uniquely quote-mapped citations can receive credit, with page capability retained separately.

Frozen common rubric; all six complete sessions
Workflow / repetitionCanonical factsSupported answersCitation source supportCorrect abstentionAttempt accounting
Kingy workflow 1.5.5A128/30 (93.3%)2 unresolved · bounds 93.3%–100.0%28/30 (93.3%)1 unresolved · bounds 93.3%–96.7%34/35 (97.1%)0 unresolved · bounds 97.1%–97.1%10/10 (100.0%)0 unresolved · bounds 100.0%–100.0%40/40 retained0 errors · 0 timeouts
Native LM Studio PDF chatB116/30 (53.3%)13 unresolved · bounds 53.3%–96.7%14/30 (46.7%)9 unresolved · bounds 46.7%–76.7%26/34 (76.5%)3 unresolved · bounds 76.5%–85.3%9/10 (90.0%)1 unresolved · bounds 90.0%–100.0%40/40 retained0 errors · 0 timeouts
Kingy workflow 1.5.5A228/30 (93.3%)2 unresolved · bounds 93.3%–100.0%28/30 (93.3%)1 unresolved · bounds 93.3%–96.7%34/35 (97.1%)0 unresolved · bounds 97.1%–97.1%10/10 (100.0%)0 unresolved · bounds 100.0%–100.0%40/40 retained0 errors · 0 timeouts
Native LM Studio PDF chatB216/30 (53.3%)13 unresolved · bounds 53.3%–96.7%14/30 (46.7%)9 unresolved · bounds 46.7%–76.7%26/34 (76.5%)3 unresolved · bounds 76.5%–85.3%9/10 (90.0%)1 unresolved · bounds 90.0%–100.0%40/40 retained0 errors · 0 timeouts
Kingy workflow 1.5.5A328/30 (93.3%)2 unresolved · bounds 93.3%–100.0%28/30 (93.3%)1 unresolved · bounds 93.3%–96.7%34/35 (97.1%)0 unresolved · bounds 97.1%–97.1%10/10 (100.0%)0 unresolved · bounds 100.0%–100.0%40/40 retained0 errors · 0 timeouts
Native LM Studio PDF chatB316/30 (53.3%)13 unresolved · bounds 53.3%–96.7%14/30 (46.7%)9 unresolved · bounds 46.7%–76.7%26/34 (76.5%)3 unresolved · bounds 76.5%–85.3%9/10 (90.0%)1 unresolved · bounds 90.0%–100.0%40/40 retained0 errors · 0 timeouts

Errors and timeouts can overlap and are not added together. Both remain failed attempts. The frozen citation metric checks for a registered source anchor in the emitted excerpt. It does not independently guarantee that the answer states the right fact: a citation can contain the expected source anchor while the answer still names the wrong event or value. Whole-answer support requires factual correctness and source checks together. Citation denominators include emitted partial or unresolved citations. Frozen shared atomic rubric1.5.0 · Protocol frozen before fresh authorship

Qualification and unresolved judgments

The declared thresholds are at least 90% supported-answer success, 95% emitted-citation precision and 90% correct abstention in every repetition, using conservative lower bounds.

Unresolved semantic or citation judgments remain. Their possible outcomes remain in the displayed bounds; an unreviewed ambiguity does not establish a comparative winner. Qualification is an absolute threshold for this workflow and test scope. A relative score does not automatically qualify a buying recommendation.

Paired effects and uncertainty

Effects below are Kingy minus native LM Studio. The 10,000-draw sensitivity analysis resamples whole unique-question clusters and keeps the three repetitions together. The envelope describes case-resampling sensitivity on this fixed corpus. It is not a population confidence interval, p-value or market-wide ranking.

Arithmetic unresolved bounds and whole-question sensitivity
Outcome / clustersEffect bounds95% case-resampling sensitivity envelope
complete question success40 unique clusters12.5 pp to 40.0 pp0.0 pp to 55.0 pp
supported answer success30 unique clusters16.7 pp to 50.0 pp0.0 pp to 66.7 pp
canonical fact correctness30 unique clusters-3.3 pp to 46.7 pp-13.3 pp to 63.3 pp
correct abstention10 unique clusters0.0 pp to 10.0 pp0.0 pp to 30.0 pp
emitted citation precision40 unique clusters11.8 pp to 20.7 pp2.8 pp to 34.4 pp

No automatic winner is declared. Paired calculations and unresolved bounds

Complete-answer latency

Descriptive time includes each workflow's full answer pipeline. Native timing includes native preprocessing, embedding/retrieval and inference. Kingy timing includes retrieval, selection, source resolution and sufficiency checks. Import, cold start and neutral warmup are recorded separately. Median and nearest-rank p95 include retained measured failures; unavailable observations remain visible. These measurements do not establish model throughput, novice setup speed or a population speed advantage.

Seconds; descriptive observations only
SessionMeasured / unavailableMedianp95Maximum
A140 / 04.4309.1859.370
A240 / 03.8447.9828.220
A340 / 03.8257.9458.201
B140 / 06.6218.2059.458
B240 / 06.2327.6538.561
B340 / 08.48614.45915.777
Actual timing boundaries

Control provenance and preparation/transport interventions

Native preparation required an additional intervention during native B2, after 27 original responses and before original28. The candidate, corpus, model, factual prompts, settings, common scorer and answer timing boundaries remained unchanged. Earlier originals and control versions are retained. Control interruptions do not establish native app reliability.

A second intervention occurred during native B3, after five original responses and before original06. Two actual More-popup attempts failed the focused-menu acknowledgment guard before any Duplicate or question submission. A prospectively reviewed foreground prefix then continued the already open canonical menu through the entire frozen eight-second menu loop and Duplicate tail, copied verbatim, with no further popup. This was one actual Duplicate via a partial-action transport continuation, not a complete unchanged invocation of the two-action helper. The first Rename, already-loaded-model binding, frozen submission once and unchanged capture retained one original06 generation. Failed acknowledgments remain failed; no synthetic full-worker success acknowledgment was created. Frozen measurement sources and content remained unchanged. Preparation overhead is incompletely measured, and the Qwen output log does not establish zero embedding work. No uninterrupted-session, setup-speed or product-reliability advantage follows.

B3 original15 was submitted once from a fresh distinct native CID with an unrenamed human display label. One physical Duplicate created the blank. A reviewed Rename AXPress was dispatched but failed field acknowledgment; subsequent stock naming controls also failed. All attempted clicks and dispatched actions remain in the completion receipt. Every name value edit and Return commit was zero. The actual CID was 1792297506398.conversation.json; the shared display title alone was not the identity check. The exact unchanged frozen submission function required the exact prompt to persist in that same target CID, plus the existing blank/name/PDF/config, selected-title, loaded-model and unique-Send guards before its sole Send. Its successful return and the retained exact first user message supply guard evidence; no separate pre-Send snapshot was retained or is claimed. One original generation was captured with matching effective settings. No extra Duplicate after naming failures, synthetic success acknowledgment, replacement model session or question retry is claimed. This disclosed identity route grants no completion or launch exemption.

Final native B3 operational accounting

All40 normalized original responses are retained separately from operational metadata. The linked B3 metadata are public privacy projections with their private-original SHA lineage; raw native backings remain preserved locally. This closure records actual execution and grants no strict-method or launch waiver.

Unrenamed positions: 15, 16, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40. Unique-name preparation failed at 15, 16, 19; optional human Rename was deliberately omitted at 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40. Each measured chat has its own native CID and first-message identity.

Original24's captured UI-complete flag remains false. Terminal EOS and the backing record were retained. A later neutral refresh of that same CID showed Done and no Stop; the original manifest flag was not rewritten and no new question was sent.

Original33's total physical Duplicate dispatch count is unknown, with a maximum of 2. Its earlier lost result permits up to 1 unknown dispatch; one later confirmed successful new preparation produced the sole measured CID and one question/generation. The generic count1 describes that successful new preparation, not every earlier dispatch. Unknown counters remain unknown.

All40 actual CID, route and preparation-count records
Position / native CIDActual display-name status and routeDuplicate dispatchesRename edits / commitsOriginal question / generationOriginal UI-complete
11792296275354.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
21792296327368.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
31792296364551.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
41792296405071.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
51792296442928.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
61792296886639.conversation.jsonUnique human labelReviewed existing-menu Duplicate continuation; first frozen Rename succeeded11 / 11 / 1true
71792297062406.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
81792297096295.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
91792297132929.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
101792297174215.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
111792297211023.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
121792297247066.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
131792297281877.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
141792297316344.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
151792297506398.conversation.jsonShared human labelReviewed existing-menu Duplicate continuation; unrenamed distinct CID with exact same-path frozen submit10 / 01 / 1true
161792298640158.conversation.jsonShared human labelReviewed existing-menu Duplicate continuation; unrenamed distinct CID with exact same-path frozen submit10 / 01 / 1true
171792298762849.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
181792298798614.conversation.jsonUnique human labelFrozen native coordinator Duplicate and unique Rename11 / 11 / 1true
191792298972925.conversation.jsonShared human labelAdditional observed stock current-menu Duplicate after reviewed wrapper stopped before Duplicate; unrenamed CID10 / 01 / 1true
201792299631870.conversation.jsonShared human labelObserved CUA stock Duplicate with delayed native effect; same existing CID reconciled, optional Rename omitted10 / 01 / 1true
211792299874315.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
221792299950492.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
231792299981056.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
241792300013094.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1false
251792300105244.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
261792300195848.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
271792300227565.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
281792300254491.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
291792300316132.conversation.jsonShared human labelObserved CUA stock Duplicate after native stock source stopped before Duplicate; optional Rename omitted10 / 01 / 1true
301792300373770.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
311792300408366.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
321792300443532.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
331792300686517.conversation.jsonShared human labelNew independently reviewed stock preparation after stable no-new-native-effect reconciliation of LOST prior result; optional Rename omittedUnknown (maximum 2)0 / 01 / 1true
341792300801494.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
351792300825517.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
361792300854585.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
371792300888860.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
381792300915019.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
391792300941132.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true
401792300972242.conversation.jsonShared human labelPost-start stock canonical Duplicate preparation amendment; optional Rename omitted10 / 01 / 1true

B3 public privacy projection: public-operational-summary-v2.json; private original and original SHA preserved · Final B3 public projection/source inventory; private original SHA lineage, not raw native exports · Independent post-start preparation transport assessment; strict method adherence separately adjudicated

The original native-control source and settings were reviewed and frozen before fresh corpus authorship. All37 frozen files were preserved, while executed native preparation transport was amended after collection began. Source-hash preservation does not establish unchanged executed control paths or strict prospective-method adherence; the independent eligibility review adjudicates that separate gate. Every native measured original uses a fresh chat with a distinct actual native CID/path and the question as its first user message. Human display-name uniqueness is not claimed for every native chat. After the B2 original09 acknowledgment-read failure, read-only reconciliation verified its existing blank chat before the frozen loaded-model selector and ui.submit paths made one submission; the failed acknowledgment was preserved, with no new Duplicate action. Before B2 original28, an actual first preparation Duplicate produced an invalid attachment copy. Supported removal attempts left a missing-backing orphan; that zero-message, unsubmitted draft and all failed actions remain privately preserved. Following prospective independent review, one further actual preparation Duplicate used the complete frozen coordinator/worker path. A separate foreground-failure request had zero Duplicate or Send actions; a neutral owned-window AXRaise and foreground assignment restored the observed foreground binding. The final measured chat had the exact five frozen PDF bytes, all five effective-setting groups and one original28 Send/generation. Thus original28 had two preparation Duplicates and one question submission, not a question retry. Separately, before B3 original06, two real More-popup AXPress attempts failed the focused-menu acknowledgment guard before any Duplicate or question submission. Following prospective independent review, a normal owned-foreground prefix preceded the entire frozen eight-second menu loop and Duplicate tail copied verbatim, continuing the known open canonical menu without another popup. This partial-action continuation issued one actual Duplicate; it was not a complete unchanged invocation of the two-action helper. The exact first Rename, already-loaded-model selector, exported frozen ui.submit once and unchanged coordinator capture then retained one original06 generation. Its measured chat had the exact five frozen PDF bytes and all five effective-setting groups. Failed worker acknowledgments remain failed; no synthetic full-worker success acknowledgment was created. Before B3 original15, one physical Duplicate created a fresh blank. Frozen and prospectively reviewed naming actions then failed: one reviewed Rename AXPress was dispatched but its field acknowledgment failed, and further stock naming controls also failed. Every attempted click and dispatched native action remains separately counted in the completion receipt; all name value edits and Return commits were zero. The chat retained its shared human display label. A prospective source/protocol review permitted the exact unchanged ui.submit with that actual label and its distinct native CID. Successful frozen ui.submit enforced empty/name/PDF/config checks and exact prompt persistence in that same CID before its sole Send; the retained first user message exactly matches the frozen prompt hash. No separate pre-Send snapshot was retained or is claimed. One original15 submission/generation and unchanged capture are retained, with no further Duplicate after naming failures. The final B3 public closure now lists every actual unrenamed position15,16 and19 through40, the per-case control versions and action receipts, original24 UI reconciliation and original33 unknown dispatch counters. These public metadata are privacy projections with private-original SHA lineage. No frozen method source, model session, load, warmup, prompt, inference helper, scorer or answer-timing boundary changed. Additional UI/import/embedding/setup overhead is unknown beyond the retained observations; the Qwen generation log does not establish zero embedding work. These preparation and transport interventions do not establish uninterrupted operation, setup speed or competitor product unreliability. Final independent completion, source, privacy and launch gates still apply.

Exact-final native neutral Duplicate and uniqueRename proof · Actual native five-PDF template proof · Independent engineering review and exact source pins · All six session provenance checks · Independent post-start preparation transport assessment; strict method adherence separately adjudicated · Independent review of existing-clone recovery · B2 public CID-basename privacy projection: original09-operational-recovery-completion.json; private-original SHA retained · Independent initial review of unsubmitted original28 preparation · Independent prospective review of one additional preparation Duplicate · B2 original28-additional-preparation-completion · B2 original28-neutral-axraise-foreground-preflight · B2 input-backing-integrity-all-native-originals · B2 effective-settings-all-originals · B2 timing-all-originals · B2 session-summary · B2 original restricted/partial post-session host capture; preserved unchanged · B2 later actual complete host-command capture; not backdated; thermal status unavailable · Pre-final independent B1/B2 metadata review; original-content and timing audit deferred · Preserved initial unexecuted B3 original06 recovery proposal review; superseded before action · Independent prospective B3 original06 partial-action transport review; no final eligibility waiver · B3 public privacy projection: original06-operational-recovery.json; private original and original SHA preserved · B3 public privacy projection: original06-operational-recovery-completion.json; private original and original SHA preserved · Prospective repeated-pattern Duplicate transport review; per-case guards still required · Prospective repeated-pattern Rename review; attempted actions remain separately counted · Prospective original15 stock naming preparation review; no inference or gate waiver · Public privacy derivative of exact-path identity review; original SHA bound, local directory withheld · B3 public privacy projection: original15-unrenamed-operational-recovery.json; private original and original SHA preserved · B3 public privacy projection: original15-unrenamed-recovery-completion.json; private original and original SHA preserved · B3 public privacy projection: public-operational-summary-v2.json; private original and original SHA preserved · Final B3 public projection/source inventory; private original SHA lineage, not raw native exports

Exact configurations and failures

Kingy settings
{
  "candidateVersion": "1.5.5",
  "commonScorerVersion": "1.5.0",
  "modelSha256": "7485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5",
  "generationSeed": 271828,
  "deadlineSeconds": 90,
  "runtime": "llama.cpp b11429",
  "sessions": {
    "A1": {
      "executor_version": "kingy-comparison-session/1.0.0",
      "system": "A",
      "repetition": 1,
      "status": "complete",
      "qualificationGranted": false,
      "complete_query_deadline_seconds": 90,
      "generation_seed": 271828,
      "client_offline_profile": "offline.sb",
      "client_offline_profile_sha256": "e5462ef1daae3785012cb8e4fa05b89105bf4ac4d98be9580a8c357365a4f54f",
      "timing_boundary": "runner.ask entry through its complete returned answer, source resolution and sufficiency check; excludes prior PDF import/index, cold start, warmup and later grading.",
      "cache_policy": "cache_prompt false; no claim that OS file caches were flushed.",
      "freeze_sha256": "87fcd2d85a9505f48b0e916ccec56b530816ee23d392ab17e5fa1a4731d14ab3",
      "schedule_sha256": "aba82fcfc544a7787964b755b5a562c5355eff50b21a3180ea92f3e104543428",
      "questions_sha256": "1fff76507bf75696b93e184d0af92291f7f289bb42880f6c7073d423345faf93",
      "executor_sha256": "e21d7c97bc87407370f87ae3184d4d247998dc6d4cbee024d7ba225186a68909",
      "runner_sha256": "ac8d1ccb48ebe017f7e2794392598967695ef3186fcf4d3889dfd579e8479458",
      "model_sha256": "7485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5",
      "corpus_pdf_sha256": {
        "ambercliff-workshops.pdf": "42127c8d71db141ea5d78305b6ecf2ebe90835b699a0a89d2bd87c2d17e82b38",
        "kelpbridge-courier.pdf": "bda885107c65224845b7130dc9a5d63c04ae847f7966caf8ff7c0f921bde7bff",
        "mossfall-invoices.pdf": "ee307474b921be4a30a21164ea5be75705235b962cd2d6c021df088b7bb38f49",
        "rillhaven-access.pdf": "2df554da00fcaf537a78e036ddd2631bcd373c89c441f13046933ae13262d850",
        "silvermere-records.pdf": "557ef50b8445bdf8e6eb90d8af2cf49727267db2a14d5e31dcd0a8ab2fd24ffa"
      },
      "planned_question_ids": [
        "v20-q07",
        "v20-q33",
        "v20-q05",
        "v20-q39",
        "v20-q36",
        "v20-q29",
        "v20-q16",
        "v20-q13",
        "v20-q28",
        "v20-q22",
        "v20-q09",
        "v20-q35",
        "v20-q26",
        "v20-q14",
        "v20-q25",
        "v20-q21",
        "v20-q04",
        "v20-q02",
        "v20-q24",
        "v20-q38",
        "v20-q01",
        "v20-q40",
        "v20-q37",
        "v20-q34",
        "v20-q20",
        "v20-q31",
        "v20-q10",
        "v20-q30",
        "v20-q08",
        "v20-q11",
        "v20-q17",
        "v20-q23",
        "v20-q18",
        "v20-q32",
        "v20-q15",
        "v20-q03",
        "v20-q12",
        "v20-q19",
        "v20-q27",
        "v20-q06"
      ],
      "session_order": [
        "A1",
        "B1",
        "B2",
        "A2",
        "A3",
        "B3"
      ],
      "artifact_verification_seconds": 1.870380499996827,
      "runtime": {
        "name": "llama.cpp",
        "binary_sha256": "51afbf4ea6485b9ada19e99af97181f12baca42ef704b1e291c26a9ac0644eae",
        "version": {
          "command": [
            "/opt/homebrew/Cellar/llama.cpp/0.6.0/bin/llama-server",
            "--version"
          ],
          "exit_code": 0,
          "stdout": "",
          "stderr": "version: 0.6.0 (build 11429, commit d81235049)\nbuilt with AppleClang 21.0.0.21000334 for Darwin arm64"
        }
      },
      "before": {
        "os": {
          "command": [
            "sw_vers"
          ],
          "exit_code": 0,
          "stdout": "ProductName:\t\tmacOS\nProductVersion:\t\t26.3\nBuildVersion:\t\t25D125",
          "stderr": ""
        },
        "hardware": {
          "command": [
            "sysctl",
            "hw.model",
            "hw.memsize",
            "hw.physicalcpu",
            "hw.logicalcpu"
          ],
          "exit_code": 0,
          "stdout": "hw.model: Mac16,7\nhw.memsize: 25769803776\nhw.physicalcpu: 14\nhw.logicalcpu: 14",
          "stderr": ""
        },
        "memory_pressure": {
          "command": [
            "memory_pressure",
            "-Q"
          ],
          "exit_code": 0,
          "stdout": "The system has 25769803776 (1572864 pages with a page size of 16384).\nSystem-wide memory free percentage: 61%",
          "stderr": ""
        },
        "swap": {
          "command": [
            "sysctl",
            "-n",
            "vm.swapusage"
          ],
          "exit_code": 0,
          "stdout": "total = 9216.00M  used = 7991.50M  free = 1224.50M  (encrypted)",
          "stderr": ""
        },
        "disk": {
          "command": [
            "df",
            "-h",
            "/"
          ],
          "exit_code": 0,
          "stdout": "Filesystem        Size    Used   Avail Capacity iused ifree %iused  Mounted on\n/dev/disk3s1s1   460Gi    11Gi    18Gi    39%    453k  189M    0%   /",
          "stderr": ""
        },
        "ac_power": {
          "command": [
            "pmset",
            "-g",
            "batt"
          ],
          "exit_code": 0,
          "stdout": "Now drawing from 'AC Power'\n -InternalBattery-0 (id=22347875)\t100%; charged; 0:00 remaining present: true",
          "stderr": ""
        },
        "power_mode": {
          "command": [
            "pmset",
            "-g",
            "custom"
          ],
          "exit_code": 0,
          "stdout": "Battery Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 0\n networkoversleep     0\n sleep                0\n lessbright           1\n tcpkeepalive         1\n disksleep            0\n SleepServices        0\nAC Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 1\n networkoversleep     0\n sleep                0\n tcpkeepalive         1\n disksleep            0\n SleepServices        0",
          "stderr": ""
        },
        "thermal": {
          "command": [
            "pmset",
            "-g",
            "therm"
          ],
          "exit_code": 0,
          "stdout": "Note: No thermal warning level has been recorded\nNote: No performance warning level has been recorded\nNote: No CPU power status has been recorded",
          "stderr": ""
        },
        "chip_inventory": [
          {
            "chip_type": "Apple M4 Pro",
            "physical_memory": "24 GB",
            "number_processors": "proc 14:10:4",
            "machine_model": "Mac16,7",
            "machine_name": "MacBook Pro"
          }
        ]
      },
      "task_owned_server_pid": 25103,
      "cold_process_to_health_seconds": 2.0383543749994715,
      "ready_model_identity": {
        "models": [
          {
            "name": "kingy-qwen3-4b",
            "model": "kingy-qwen3-4b",
            "modified_at": "",
            "size": "",
            "digest": "",
            "type": "model",
            "description": "",
            "tags": [
              ""
            ],
            "capabilities": [
              "completion"
            ],
            "parameters": "",
            "details": {
              "parent_model": "",
              "format": "gguf",
              "family": "",
              "families": [
                ""
              ],
              "parameter_size": "",
              "quantization_level": ""
            }
          }
        ],
        "object": "list",
        "data": [
          {
            "id": "kingy-qwen3-4b",
            "aliases": [
              "kingy-qwen3-4b"
            ],
            "tags": [],
            "object": "model",
            "architecture": {
              "input_modalities": [
                "text"
              ],
              "output_modalities": [
                "text"
              ]
            },
            "created": 1792289802,
            "owned_by": "llamacpp",
            "meta": {
              "vocab_type": 2,
              "n_vocab": 151936,
              "n_ctx": 8192,
              "n_ctx_train": 40960,
              "n_embd": 2560,
              "n_params": 4022468096,
              "size": 2491323904,
              "ftype": "Q4_K - Medium"
            }
          }
        ]
      },
      "warmup_model_request_count": 1,
      "warmup_seconds": 0.08470962499995949,
      "parse_seconds": 0.03763466599775711,
      "index_seconds": 0.0013065000021015294,
      "loaded_pdf_pages": 30,
      "retained_attempts": 40,
      "planned_attempts": 40,
      "unexecuted_question_ids": [],
      "unexecuted_attempt_count": 0,
      "model_request_count": 90,
      "session_total_model_request_count": 91,
      "attempt_errors": 0,
      "timeouts": 0,
      "after": {
        "os": {
          "command": [
            "sw_vers"
          ],
          "exit_code": 0,
          "stdout": "ProductName:\t\tmacOS\nProductVersion:\t\t26.3\nBuildVersion:\t\t25D125",
          "stderr": ""
        },
        "hardware": {
          "command": [
            "sysctl",
            "hw.model",
            "hw.memsize",
            "hw.physicalcpu",
            "hw.logicalcpu"
          ],
          "exit_code": 0,
          "stdout": "hw.model: Mac16,7\nhw.memsize: 25769803776\nhw.physicalcpu: 14\nhw.logicalcpu: 14",
          "stderr": ""
        },
        "memory_pressure": {
          "command": [
            "memory_pressure",
            "-Q"
          ],
          "exit_code": 0,
          "stdout": "The system has 25769803776 (1572864 pages with a page size of 16384).\nSystem-wide memory free percentage: 58%",
          "stderr": ""
        },
        "swap": {
          "command": [
            "sysctl",
            "-n",
            "vm.swapusage"
          ],
          "exit_code": 0,
          "stdout": "total = 11264.00M  used = 10025.12M  free = 1238.88M  (encrypted)",
          "stderr": ""
        },
        "disk": {
          "command": [
            "df",
            "-h",
            "/"
          ],
          "exit_code": 0,
          "stdout": "Filesystem        Size    Used   Avail Capacity iused ifree %iused  Mounted on\n/dev/disk3s1s1   460Gi    11Gi    16Gi    42%    453k  168M    0%   /",
          "stderr": ""
        },
        "ac_power": {
          "command": [
            "pmset",
            "-g",
            "batt"
          ],
          "exit_code": 0,
          "stdout": "Now drawing from 'AC Power'\n -InternalBattery-0 (id=22347875)\t100%; charged; 0:00 remaining present: true",
          "stderr": ""
        },
        "power_mode": {
          "command": [
            "pmset",
            "-g",
            "custom"
          ],
          "exit_code": 0,
          "stdout": "Battery Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 0\n networkoversleep     0\n sleep                0\n lessbright           1\n tcpkeepalive         1\n disksleep            0\n SleepServices        0\nAC Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 1\n networkoversleep     0\n sleep                0\n tcpkeepalive         1\n disksleep            0\n SleepServices        0",
          "stderr": ""
        },
        "thermal": {
          "command": [
            "pmset",
            "-g",
            "therm"
          ],
          "exit_code": 0,
          "stdout": "Note: No thermal warning level has been recorded\nNote: No performance warning level has been recorded\nNote: No CPU power status has been recorded",
          "stderr": ""
        },
        "chip_inventory": [
          {
            "chip_type": "Apple M4 Pro",
            "physical_memory": "24 GB",
            "number_processors": "proc 14:10:4",
            "machine_model": "Mac16,7",
            "machine_name": "MacBook Pro"
          }
        ]
      },
      "session_wall_seconds": 203.58562229199742,
      "frozen_inputs_unchanged": true
    },
    "A2": {
      "executor_version": "kingy-comparison-session/1.0.0",
      "system": "A",
      "repetition": 2,
      "status": "complete",
      "qualificationGranted": false,
      "complete_query_deadline_seconds": 90,
      "generation_seed": 271828,
      "client_offline_profile": "offline.sb",
      "client_offline_profile_sha256": "e5462ef1daae3785012cb8e4fa05b89105bf4ac4d98be9580a8c357365a4f54f",
      "timing_boundary": "runner.ask entry through its complete returned answer, source resolution and sufficiency check; excludes prior PDF import/index, cold start, warmup and later grading.",
      "cache_policy": "cache_prompt false; no claim that OS file caches were flushed.",
      "freeze_sha256": "87fcd2d85a9505f48b0e916ccec56b530816ee23d392ab17e5fa1a4731d14ab3",
      "schedule_sha256": "aba82fcfc544a7787964b755b5a562c5355eff50b21a3180ea92f3e104543428",
      "questions_sha256": "1fff76507bf75696b93e184d0af92291f7f289bb42880f6c7073d423345faf93",
      "executor_sha256": "e21d7c97bc87407370f87ae3184d4d247998dc6d4cbee024d7ba225186a68909",
      "runner_sha256": "ac8d1ccb48ebe017f7e2794392598967695ef3186fcf4d3889dfd579e8479458",
      "model_sha256": "7485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5",
      "corpus_pdf_sha256": {
        "ambercliff-workshops.pdf": "42127c8d71db141ea5d78305b6ecf2ebe90835b699a0a89d2bd87c2d17e82b38",
        "kelpbridge-courier.pdf": "bda885107c65224845b7130dc9a5d63c04ae847f7966caf8ff7c0f921bde7bff",
        "mossfall-invoices.pdf": "ee307474b921be4a30a21164ea5be75705235b962cd2d6c021df088b7bb38f49",
        "rillhaven-access.pdf": "2df554da00fcaf537a78e036ddd2631bcd373c89c441f13046933ae13262d850",
        "silvermere-records.pdf": "557ef50b8445bdf8e6eb90d8af2cf49727267db2a14d5e31dcd0a8ab2fd24ffa"
      },
      "planned_question_ids": [
        "v20-q23",
        "v20-q30",
        "v20-q02",
        "v20-q07",
        "v20-q40",
        "v20-q20",
        "v20-q28",
        "v20-q27",
        "v20-q01",
        "v20-q12",
        "v20-q24",
        "v20-q26",
        "v20-q22",
        "v20-q31",
        "v20-q19",
        "v20-q32",
        "v20-q06",
        "v20-q36",
        "v20-q03",
        "v20-q04",
        "v20-q35",
        "v20-q15",
        "v20-q16",
        "v20-q11",
        "v20-q29",
        "v20-q18",
        "v20-q09",
        "v20-q17",
        "v20-q14",
        "v20-q21",
        "v20-q05",
        "v20-q10",
        "v20-q39",
        "v20-q08",
        "v20-q25",
        "v20-q34",
        "v20-q38",
        "v20-q13",
        "v20-q37",
        "v20-q33"
      ],
      "session_order": [
        "A1",
        "B1",
        "B2",
        "A2",
        "A3",
        "B3"
      ],
      "artifact_verification_seconds": 1.914748082999722,
      "runtime": {
        "name": "llama.cpp",
        "binary_sha256": "51afbf4ea6485b9ada19e99af97181f12baca42ef704b1e291c26a9ac0644eae",
        "version": {
          "command": [
            "/opt/homebrew/Cellar/llama.cpp/0.6.0/bin/llama-server",
            "--version"
          ],
          "exit_code": 0,
          "stdout": "",
          "stderr": "version: 0.6.0 (build 11429, commit d81235049)\nbuilt with AppleClang 21.0.0.21000334 for Darwin arm64"
        }
      },
      "before": {
        "os": {
          "command": [
            "sw_vers"
          ],
          "exit_code": 0,
          "stdout": "ProductName:\t\tmacOS\nProductVersion:\t\t26.3\nBuildVersion:\t\t25D125",
          "stderr": ""
        },
        "hardware": {
          "command": [
            "sysctl",
            "hw.model",
            "hw.memsize",
            "hw.physicalcpu",
            "hw.logicalcpu"
          ],
          "exit_code": 0,
          "stdout": "hw.model: Mac16,7\nhw.memsize: 25769803776\nhw.physicalcpu: 14\nhw.logicalcpu: 14",
          "stderr": ""
        },
        "memory_pressure": {
          "command": [
            "memory_pressure",
            "-Q"
          ],
          "exit_code": 0,
          "stdout": "The system has 25769803776 (1572864 pages with a page size of 16384).\nSystem-wide memory free percentage: 76%",
          "stderr": ""
        },
        "swap": {
          "command": [
            "sysctl",
            "-n",
            "vm.swapusage"
          ],
          "exit_code": 0,
          "stdout": "total = 12288.00M  used = 10833.19M  free = 1454.81M  (encrypted)",
          "stderr": ""
        },
        "disk": {
          "command": [
            "df",
            "-h",
            "/"
          ],
          "exit_code": 0,
          "stdout": "Filesystem        Size    Used   Avail Capacity iused ifree %iused  Mounted on\n/dev/disk3s1s1   460Gi    11Gi    15Gi    44%    453k  154M    0%   /",
          "stderr": ""
        },
        "ac_power": {
          "command": [
            "pmset",
            "-g",
            "batt"
          ],
          "exit_code": 0,
          "stdout": "Now drawing from 'AC Power'\n -InternalBattery-0 (id=22347875)\t100%; charged; 0:00 remaining present: true",
          "stderr": ""
        },
        "power_mode": {
          "command": [
            "pmset",
            "-g",
            "custom"
          ],
          "exit_code": 0,
          "stdout": "Battery Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 0\n networkoversleep     0\n sleep                0\n lessbright           1\n tcpkeepalive         1\n disksleep            0\n SleepServices        0\nAC Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 1\n networkoversleep     0\n sleep                0\n tcpkeepalive         1\n disksleep            0\n SleepServices        0",
          "stderr": ""
        },
        "thermal": {
          "command": [
            "pmset",
            "-g",
            "therm"
          ],
          "exit_code": 0,
          "stdout": "Note: No thermal warning level has been recorded\nNote: No performance warning level has been recorded\nNote: No CPU power status has been recorded",
          "stderr": ""
        },
        "chip_inventory": [
          {
            "chip_type": "Apple M4 Pro",
            "physical_memory": "24 GB",
            "number_processors": "proc 14:10:4",
            "machine_model": "Mac16,7",
            "machine_name": "MacBook Pro"
          }
        ]
      },
      "task_owned_server_pid": 46494,
      "cold_process_to_health_seconds": 1.7877282920017024,
      "ready_model_identity": {
        "models": [
          {
            "name": "kingy-qwen3-4b",
            "model": "kingy-qwen3-4b",
            "modified_at": "",
            "size": "",
            "digest": "",
            "type": "model",
            "description": "",
            "tags": [
              ""
            ],
            "capabilities": [
              "completion"
            ],
            "parameters": "",
            "details": {
              "parent_model": "",
              "format": "gguf",
              "family": "",
              "families": [
                ""
              ],
              "parameter_size": "",
              "quantization_level": ""
            }
          }
        ],
        "object": "list",
        "data": [
          {
            "id": "kingy-qwen3-4b",
            "aliases": [
              "kingy-qwen3-4b"
            ],
            "tags": [],
            "object": "model",
            "architecture": {
              "input_modalities": [
                "text"
              ],
              "output_modalities": [
                "text"
              ]
            },
            "created": 1792295605,
            "owned_by": "llamacpp",
            "meta": {
              "vocab_type": 2,
              "n_vocab": 151936,
              "n_ctx": 8192,
              "n_ctx_train": 40960,
              "n_embd": 2560,
              "n_params": 4022468096,
              "size": 2491323904,
              "ftype": "Q4_K - Medium"
            }
          }
        ]
      },
      "warmup_model_request_count": 1,
      "warmup_seconds": 0.06547408299957169,
      "parse_seconds": 0.024295957999129314,
      "index_seconds": 0.000991875000181608,
      "loaded_pdf_pages": 30,
      "retained_attempts": 40,
      "planned_attempts": 40,
      "unexecuted_question_ids": [],
      "unexecuted_attempt_count": 0,
      "model_request_count": 90,
      "session_total_model_request_count": 91,
      "attempt_errors": 0,
      "timeouts": 0,
      "after": {
        "os": {
          "command": [
            "sw_vers"
          ],
          "exit_code": 0,
          "stdout": "ProductName:\t\tmacOS\nProductVersion:\t\t26.3\nBuildVersion:\t\t25D125",
          "stderr": ""
        },
        "hardware": {
          "command": [
            "sysctl",
            "hw.model",
            "hw.memsize",
            "hw.physicalcpu",
            "hw.logicalcpu"
          ],
          "exit_code": 0,
          "stdout": "hw.model: Mac16,7\nhw.memsize: 25769803776\nhw.physicalcpu: 14\nhw.logicalcpu: 14",
          "stderr": ""
        },
        "memory_pressure": {
          "command": [
            "memory_pressure",
            "-Q"
          ],
          "exit_code": 0,
          "stdout": "The system has 25769803776 (1572864 pages with a page size of 16384).\nSystem-wide memory free percentage: 71%",
          "stderr": ""
        },
        "swap": {
          "command": [
            "sysctl",
            "-n",
            "vm.swapusage"
          ],
          "exit_code": 0,
          "stdout": "total = 12288.00M  used = 10737.19M  free = 1550.81M  (encrypted)",
          "stderr": ""
        },
        "disk": {
          "command": [
            "df",
            "-h",
            "/"
          ],
          "exit_code": 0,
          "stdout": "Filesystem        Size    Used   Avail Capacity iused ifree %iused  Mounted on\n/dev/disk3s1s1   460Gi    11Gi    15Gi    44%    453k  154M    0%   /",
          "stderr": ""
        },
        "ac_power": {
          "command": [
            "pmset",
            "-g",
            "batt"
          ],
          "exit_code": 0,
          "stdout": "Now drawing from 'AC Power'\n -InternalBattery-0 (id=22347875)\t100%; charged; 0:00 remaining present: true",
          "stderr": ""
        },
        "power_mode": {
          "command": [
            "pmset",
            "-g",
            "custom"
          ],
          "exit_code": 0,
          "stdout": "Battery Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 0\n networkoversleep     0\n sleep                0\n lessbright           1\n tcpkeepalive         1\n disksleep            0\n SleepServices        0\nAC Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 1\n networkoversleep     0\n sleep                0\n tcpkeepalive         1\n disksleep            0\n SleepServices        0",
          "stderr": ""
        },
        "thermal": {
          "command": [
            "pmset",
            "-g",
            "therm"
          ],
          "exit_code": 0,
          "stdout": "Note: No thermal warning level has been recorded\nNote: No performance warning level has been recorded\nNote: No CPU power status has been recorded",
          "stderr": ""
        },
        "chip_inventory": [
          {
            "chip_type": "Apple M4 Pro",
            "physical_memory": "24 GB",
            "number_processors": "proc 14:10:4",
            "machine_model": "Mac16,7",
            "machine_name": "MacBook Pro"
          }
        ]
      },
      "session_wall_seconds": 177.1890857920007,
      "frozen_inputs_unchanged": true
    },
    "A3": {
      "executor_version": "kingy-comparison-session/1.0.0",
      "system": "A",
      "repetition": 3,
      "status": "complete",
      "qualificationGranted": false,
      "complete_query_deadline_seconds": 90,
      "generation_seed": 271828,
      "client_offline_profile": "offline.sb",
      "client_offline_profile_sha256": "e5462ef1daae3785012cb8e4fa05b89105bf4ac4d98be9580a8c357365a4f54f",
      "timing_boundary": "runner.ask entry through its complete returned answer, source resolution and sufficiency check; excludes prior PDF import/index, cold start, warmup and later grading.",
      "cache_policy": "cache_prompt false; no claim that OS file caches were flushed.",
      "freeze_sha256": "87fcd2d85a9505f48b0e916ccec56b530816ee23d392ab17e5fa1a4731d14ab3",
      "schedule_sha256": "aba82fcfc544a7787964b755b5a562c5355eff50b21a3180ea92f3e104543428",
      "questions_sha256": "1fff76507bf75696b93e184d0af92291f7f289bb42880f6c7073d423345faf93",
      "executor_sha256": "e21d7c97bc87407370f87ae3184d4d247998dc6d4cbee024d7ba225186a68909",
      "runner_sha256": "ac8d1ccb48ebe017f7e2794392598967695ef3186fcf4d3889dfd579e8479458",
      "model_sha256": "7485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5",
      "corpus_pdf_sha256": {
        "ambercliff-workshops.pdf": "42127c8d71db141ea5d78305b6ecf2ebe90835b699a0a89d2bd87c2d17e82b38",
        "kelpbridge-courier.pdf": "bda885107c65224845b7130dc9a5d63c04ae847f7966caf8ff7c0f921bde7bff",
        "mossfall-invoices.pdf": "ee307474b921be4a30a21164ea5be75705235b962cd2d6c021df088b7bb38f49",
        "rillhaven-access.pdf": "2df554da00fcaf537a78e036ddd2631bcd373c89c441f13046933ae13262d850",
        "silvermere-records.pdf": "557ef50b8445bdf8e6eb90d8af2cf49727267db2a14d5e31dcd0a8ab2fd24ffa"
      },
      "planned_question_ids": [
        "v20-q18",
        "v20-q16",
        "v20-q06",
        "v20-q05",
        "v20-q37",
        "v20-q23",
        "v20-q28",
        "v20-q04",
        "v20-q26",
        "v20-q22",
        "v20-q34",
        "v20-q32",
        "v20-q13",
        "v20-q35",
        "v20-q07",
        "v20-q20",
        "v20-q24",
        "v20-q29",
        "v20-q02",
        "v20-q14",
        "v20-q39",
        "v20-q38",
        "v20-q09",
        "v20-q36",
        "v20-q17",
        "v20-q01",
        "v20-q25",
        "v20-q11",
        "v20-q19",
        "v20-q30",
        "v20-q03",
        "v20-q27",
        "v20-q10",
        "v20-q12",
        "v20-q15",
        "v20-q33",
        "v20-q08",
        "v20-q21",
        "v20-q40",
        "v20-q31"
      ],
      "session_order": [
        "A1",
        "B1",
        "B2",
        "A2",
        "A3",
        "B3"
      ],
      "artifact_verification_seconds": 1.8542495000001509,
      "runtime": {
        "name": "llama.cpp",
        "binary_sha256": "51afbf4ea6485b9ada19e99af97181f12baca42ef704b1e291c26a9ac0644eae",
        "version": {
          "command": [
            "/opt/homebrew/Cellar/llama.cpp/0.6.0/bin/llama-server",
            "--version"
          ],
          "exit_code": 0,
          "stdout": "",
          "stderr": "version: 0.6.0 (build 11429, commit d81235049)\nbuilt with AppleClang 21.0.0.21000334 for Darwin arm64"
        }
      },
      "before": {
        "os": {
          "command": [
            "sw_vers"
          ],
          "exit_code": 0,
          "stdout": "ProductName:\t\tmacOS\nProductVersion:\t\t26.3\nBuildVersion:\t\t25D125",
          "stderr": ""
        },
        "hardware": {
          "command": [
            "sysctl",
            "hw.model",
            "hw.memsize",
            "hw.physicalcpu",
            "hw.logicalcpu"
          ],
          "exit_code": 0,
          "stdout": "hw.model: Mac16,7\nhw.memsize: 25769803776\nhw.physicalcpu: 14\nhw.logicalcpu: 14",
          "stderr": ""
        },
        "memory_pressure": {
          "command": [
            "memory_pressure",
            "-Q"
          ],
          "exit_code": 0,
          "stdout": "The system has 25769803776 (1572864 pages with a page size of 16384).\nSystem-wide memory free percentage: 73%",
          "stderr": ""
        },
        "swap": {
          "command": [
            "sysctl",
            "-n",
            "vm.swapusage"
          ],
          "exit_code": 0,
          "stdout": "total = 12288.00M  used = 10737.19M  free = 1550.81M  (encrypted)",
          "stderr": ""
        },
        "disk": {
          "command": [
            "df",
            "-h",
            "/"
          ],
          "exit_code": 0,
          "stdout": "Filesystem        Size    Used   Avail Capacity iused ifree %iused  Mounted on\n/dev/disk3s1s1   460Gi    11Gi    15Gi    44%    453k  154M    0%   /",
          "stderr": ""
        },
        "ac_power": {
          "command": [
            "pmset",
            "-g",
            "batt"
          ],
          "exit_code": 0,
          "stdout": "Now drawing from 'AC Power'\n -InternalBattery-0 (id=22347875)\t100%; charged; 0:00 remaining present: true",
          "stderr": ""
        },
        "power_mode": {
          "command": [
            "pmset",
            "-g",
            "custom"
          ],
          "exit_code": 0,
          "stdout": "Battery Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 0\n networkoversleep     0\n sleep                0\n lessbright           1\n tcpkeepalive         1\n disksleep            0\n SleepServices        0\nAC Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 1\n networkoversleep     0\n sleep                0\n tcpkeepalive         1\n disksleep            0\n SleepServices        0",
          "stderr": ""
        },
        "thermal": {
          "command": [
            "pmset",
            "-g",
            "therm"
          ],
          "exit_code": 0,
          "stdout": "Note: No thermal warning level has been recorded\nNote: No performance warning level has been recorded\nNote: No CPU power status has been recorded",
          "stderr": ""
        },
        "chip_inventory": [
          {
            "chip_type": "Apple M4 Pro",
            "physical_memory": "24 GB",
            "number_processors": "proc 14:10:4",
            "machine_model": "Mac16,7",
            "machine_name": "MacBook Pro"
          }
        ]
      },
      "task_owned_server_pid": 47027,
      "cold_process_to_health_seconds": 1.55356645899883,
      "ready_model_identity": {
        "models": [
          {
            "name": "kingy-qwen3-4b",
            "model": "kingy-qwen3-4b",
            "modified_at": "",
            "size": "",
            "digest": "",
            "type": "model",
            "description": "",
            "tags": [
              ""
            ],
            "capabilities": [
              "completion"
            ],
            "parameters": "",
            "details": {
              "parent_model": "",
              "format": "gguf",
              "family": "",
              "families": [
                ""
              ],
              "parameter_size": "",
              "quantization_level": ""
            }
          }
        ],
        "object": "list",
        "data": [
          {
            "id": "kingy-qwen3-4b",
            "aliases": [
              "kingy-qwen3-4b"
            ],
            "tags": [],
            "object": "model",
            "architecture": {
              "input_modalities": [
                "text"
              ],
              "output_modalities": [
                "text"
              ]
            },
            "created": 1792295797,
            "owned_by": "llamacpp",
            "meta": {
              "vocab_type": 2,
              "n_vocab": 151936,
              "n_ctx": 8192,
              "n_ctx_train": 40960,
              "n_embd": 2560,
              "n_params": 4022468096,
              "size": 2491323904,
              "ftype": "Q4_K - Medium"
            }
          }
        ]
      },
      "warmup_model_request_count": 1,
      "warmup_seconds": 0.06496091699955286,
      "parse_seconds": 0.02428633299859939,
      "index_seconds": 0.000983291000011377,
      "loaded_pdf_pages": 30,
      "retained_attempts": 40,
      "planned_attempts": 40,
      "unexecuted_question_ids": [],
      "unexecuted_attempt_count": 0,
      "model_request_count": 90,
      "session_total_model_request_count": 91,
      "attempt_errors": 0,
      "timeouts": 0,
      "after": {
        "os": {
          "command": [
            "sw_vers"
          ],
          "exit_code": 0,
          "stdout": "ProductName:\t\tmacOS\nProductVersion:\t\t26.3\nBuildVersion:\t\t25D125",
          "stderr": ""
        },
        "hardware": {
          "command": [
            "sysctl",
            "hw.model",
            "hw.memsize",
            "hw.physicalcpu",
            "hw.logicalcpu"
          ],
          "exit_code": 0,
          "stdout": "hw.model: Mac16,7\nhw.memsize: 25769803776\nhw.physicalcpu: 14\nhw.logicalcpu: 14",
          "stderr": ""
        },
        "memory_pressure": {
          "command": [
            "memory_pressure",
            "-Q"
          ],
          "exit_code": 0,
          "stdout": "The system has 25769803776 (1572864 pages with a page size of 16384).\nSystem-wide memory free percentage: 69%",
          "stderr": ""
        },
        "swap": {
          "command": [
            "sysctl",
            "-n",
            "vm.swapusage"
          ],
          "exit_code": 0,
          "stdout": "total = 12288.00M  used = 10617.19M  free = 1670.81M  (encrypted)",
          "stderr": ""
        },
        "disk": {
          "command": [
            "df",
            "-h",
            "/"
          ],
          "exit_code": 0,
          "stdout": "Filesystem        Size    Used   Avail Capacity iused ifree %iused  Mounted on\n/dev/disk3s1s1   460Gi    11Gi    15Gi    44%    453k  154M    0%   /",
          "stderr": ""
        },
        "ac_power": {
          "command": [
            "pmset",
            "-g",
            "batt"
          ],
          "exit_code": 0,
          "stdout": "Now drawing from 'AC Power'\n -InternalBattery-0 (id=22347875)\t100%; charged; 0:00 remaining present: true",
          "stderr": ""
        },
        "power_mode": {
          "command": [
            "pmset",
            "-g",
            "custom"
          ],
          "exit_code": 0,
          "stdout": "Battery Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 0\n networkoversleep     0\n sleep                0\n lessbright           1\n tcpkeepalive         1\n disksleep            0\n SleepServices        0\nAC Power:\n Sleep On Power Button 1\n powermode            0\n standby              0\n ttyskeepawake        1\n hibernatemode        3\n powernap             0\n hibernatefile        /var/vm/sleepimage\n displaysleep         0\n womp                 1\n networkoversleep     0\n sleep                0\n tcpkeepalive         1\n disksleep            0\n SleepServices        0",
          "stderr": ""
        },
        "thermal": {
          "command": [
            "pmset",
            "-g",
            "therm"
          ],
          "exit_code": 0,
          "stdout": "Note: No thermal warning level has been recorded\nNote: No performance warning level has been recorded\nNote: No CPU power status has been recorded",
          "stderr": ""
        },
        "chip_inventory": [
          {
            "chip_type": "Apple M4 Pro",
            "physical_memory": "24 GB",
            "number_processors": "proc 14:10:4",
            "machine_model": "Mac16,7",
            "machine_name": "MacBook Pro"
          }
        ]
      },
      "session_wall_seconds": 176.2300627920049,
      "frozen_inputs_unchanged": true
    }
  }
}

Candidate/model/seed/deadline and all original session metadata

Native LM Studio settings
{
  "load_config": {
    "fields": [
      {
        "key": "llm.load.llama.autoFit",
        "value": false
      },
      {
        "key": "llm.load.mlx.autoFit",
        "value": false
      },
      {
        "key": "llm.load.llama.cpuThreadPoolSize",
        "value": 10
      },
      {
        "key": "llm.load.contextLength",
        "value": 8192
      },
      {
        "key": "llm.load.llama.flashAttention",
        "value": true
      },
      {
        "key": "llm.load.promptTemplate",
        "value": {
          "type": "jinja",
          "jinjaPromptTemplate": {
            "template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for index in range(ns.last_query_index, -1, -1) %}\n    {%- set message = messages[index] %}\n    {%- if ns.multi_step_tool and message.role == \"user\" and not('<tool_response>' in message.content and '</tool_response>' in message.content) %}\n        {%- set ns.multi_step_tool = false %}\n        {%- set ns.last_query_index = index %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set content = message.content %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in message.content %}\n                {%- set content = message.content.split('</think>')[-1].lstrip('\\n') %}\n                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- endif %}\n{%- endif %}"
          }
        }
      },
      {
        "key": "llm.load.llama.tryDirectIO",
        "value": false
      },
      {
        "key": "llm.load.llama.acceleration.offloadRatio",
        "value": 1
      },
      {
        "key": "llm.load.numParallelSessions",
        "value": 1
      },
      {
        "key": "llm.load.seed",
        "value": {
          "checked": true,
          "value": 271828
        }
      },
      {
        "key": "llm.load.yuzu.autoFit",
        "value": false
      },
      {
        "key": "llm.load.vllm.autoFit",
        "value": false
      }
    ]
  },
  "prediction_config": {
    "fields": [
      {
        "key": "llm.prediction.promptTemplate",
        "value": {
          "type": "jinja",
          "jinjaPromptTemplate": {
            "template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for index in range(ns.last_query_index, -1, -1) %}\n    {%- set message = messages[index] %}\n    {%- if ns.multi_step_tool and message.role == \"user\" and not('<tool_response>' in message.content and '</tool_response>' in message.content) %}\n        {%- set ns.multi_step_tool = false %}\n        {%- set ns.last_query_index = index %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set content = message.content %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in message.content %}\n                {%- set content = message.content.split('</think>')[-1].lstrip('\\n') %}\n                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- endif %}\n{%- endif %}"
          },
          "stopStrings": []
        }
      },
      {
        "key": "llm.prediction.llama.cpuThreads",
        "value": 10
      },
      {
        "key": "llm.prediction.repeatPenalty",
        "value": {
          "checked": true,
          "value": 1
        }
      },
      {
        "key": "llm.prediction.topPSampling",
        "value": {
          "checked": true,
          "value": 1
        }
      },
      {
        "key": "llm.prediction.minPSampling",
        "value": {
          "checked": true,
          "value": 0
        }
      },
      {
        "key": "llm.prediction.temperature",
        "value": 0
      },
      {
        "key": "llm.prediction.maxPredictedTokens",
        "value": {
          "checked": true,
          "value": 512
        }
      },
      {
        "key": "llm.prediction.reasoning.enableThinking",
        "value": false
      },
      {
        "key": "llm.prediction.contextOverflowPolicy",
        "value": "stopAtLimit"
      },
      {
        "key": "llm.prediction.tools",
        "value": {
          "type": "none"
        }
      }
    ]
  },
  "plugins": [
    "lmstudio/rag-v1"
  ],
  "plugin_configs": {
    "lmstudio/rag-v1": {
      "preset": "",
      "config": {
        "fields": [
          {
            "key": "retrievalLimit",
            "value": 8
          }
        ]
      }
    }
  },
  "system_prompt": ""
}

Actual effective native settings

Fresh installed engine/embedding/app artifact checks · Official native RAG artifact pins · Complete method/candidate/corpus/native-template freeze · All six session provenance checks

The retained attempt records contain no runtime errors or timeouts. Quality failures and unresolved judgments remain in the score denominators and full raw evidence.

All120 Kingy originals and frozen grades · All120 normalized native original responses and frozen grades; private backing SHA lineage retained

Limits and history

The original immutable v1.4 paired pilot remains separate historical evidence, including its incomplete native repetitions and post-output source review. Its scores and files are preserved. The original failed 1.5.0 study and its scorer limitation also remain unchanged. The original failed 1.5.1 study also remains unchanged. The preserved v19 study remains unchanged, including its scoring defects and incomplete native original. This candidate1.5.5 report uses its own frozen rubric and fresh study; it does not rewrite earlier comparisons. Original immutable1.4 manifest · Original1.5.0 manifest

Inspect and reproduce

Report data · File hashes and sizes · Analysis completion and exact output hashes