# Shared v1.5 document scoring policy

This method is prepared before any fresh v1.5 bank content or output access by the scorer/repair agents. Development uses consumed v1.4 results, its retained supplemental source audit, and synthetic controls. All original v1.4 scores, code and evidence remain unchanged. This is a new preregistered rubric; development-control gains are not a held-out improvement claim.

## Systems and blinded inputs

Compare the complete Kingy factual-document pipeline with stock LM Studio native document chat. Both receive the same ten PDF bytes, question and required field labels. Neither receives reference facts, accepted alternatives or evidence spans. Use the same pinned model artifact, machine, context, temperature, seed and declared resolved settings; record actual runtime/backend and native retrieval/embedding differences. Only original first-message native chats are used. The final settings and inference/capture sources must be frozen separately by the experiment owner. This scorer does not start models, read bank files, load a network client or decide how a model answers.

## Reference contract and concise values

References contain required_fields, expected_abstention, answer_fields, essential_claims and optional accepted_alternatives. Each essential claim contains field, a compact semantically complete value, and exact document/page evidence spans. accepted_alternatives is a field-to-list mapping of complete equivalent strings authored before outputs; legacy per-claim accepted_values remains supported. Never add an alias or support span to rescue a consumed new-bank output. Unknown references may include abstention_evidence/absence_audit for author/source audit; no such data enters inference or format adaptation.

The complete answer must supply every requested value, number, sign, unit, time basis, event, stage, condition and exclusion. Compact values may omit redundant scope wording established by the question or exact typed field. Limit/minimum wording can be implicit when the question/field explicitly asks that bound; an explicit opposite direction or exact-required quantity is not equivalent. A final-inspection field label can state that exact stage, while a generic width field or question merely asking for a stage cannot. Explicit contrary stage text fails. Per-batch/per-time bases, calendar/working-day distinctions, actual dates, beginning/ending events, written/other acceptance and before/after/until conditions are never supplied by question context. Complete source sentences and exact source passages remain accepted only when they contain the essential fact and have linked support.

Context selection uses the question clause most clearly associated with the field. No context can supply a missing numeric value, unit, date/month/day or identifier. Scope shortening is narrow: a reference's trailing subject phrase introduced by of/for/between/in/on, or dispatch-batch scope following actual named weekdays, must already be explicit in the selected question/field. Event/condition/basis words and actual weekdays/months cannot be removed. Unknown or ambiguous semantic equivalents remain unresolved.

## Deterministic normalization and output structure

Unicode/ordinary whitespace and punctuation are cosmetic. Numeric minus/plus, decimals, ambiguous commas, identifiers, comparisons and qualifiers remain meaningful. Standalone cardinal words zero through twenty and a closed list of unit spellings/abbreviations normalize without arithmetic or unit conversion. Common plural forms and inspected/inspection, require/requires and recheck/rechecked variants normalize. Generic article a/an/the/any is cosmetic in a policy/value phrase; a symbol immediately following a numeric value is retained. Hyphenated identifiers/compounds are not globally rewritten; equivalent complete prose variants belong in frozen aliases. Inclusive at most/up to/no more than and at least/no less than comparisons normalize separately from strict greater/longer or less/shorter comparisons.

Both normalized fields/citations and native prose use the same grade_response. The native adapter knows requested labels and retained native citation metadata, never gold values. A single field can use its full prose; multiple fields need explicit labels. Unassigned prefaces, duplicate/missing labels and extra output are retained for review. Complete field-labelled UNKNOWN lines are explicit abstention only when every field occurs once, with no guessed values, extra prose or citations. Raw model prose is preserved.

Parsed native field labels are structure, not additional factual assertions in absence checking. Their complete values still retain extra prose. Additional assertions must be grounded in exact source passages/declared spans, complete copied sentences, complete frozen value/alias variants or neutral question/field subject wording. Borrowing an ordered bag of source words cannot silently create a new factual assertion or remove a condition. Unsupported numbers, explicit wrong units/direction/identifiers, mixed missing fields and supplied UNKNOWN for answerable fields fail; unregistered semantic compositions remain unresolved.

## Evidence and abstention

Only citations explicitly emitted by an answer count. Native retrieval cards alone are not answer citations. Retained IDs must map unambiguously to exact quotes and source PDF names/hashes; the capture process proves those metadata relationships separately. Quotes must ground in the actual document text. Unique quote-to-page mapping receives support credit while explicit page formatting is a separate capability. Ambiguous repeated quote locations remain unresolved. A wrong supplied page fails. Exact declared spans and original character offsets remain authoritative. The existing narrowly documented synthetic filename/Page footer whitespace repair is retained symmetrically; it does not normalize arbitrary word/number joins.

A citation receives pass only when it contains a preregistered exact supporting span. A different grounded passage in the same document containing the canonical fact can be plausible alternate evidence, but stays unresolved until separate source review; it cannot receive an automatic pass. Wrong documents, wrong facts, fabricated quotes and missing citations fail. Missing acceptance/source context cannot become proven support merely because an identifier appears.

A clean explicit decline such as UNKNOWN passes unanswerable questions. Exact source-grounded documentary-absence prose can pass after field-label removal when its topic and named-source scope match and it states no guessed requested value. A factual zero/no-discount/no-charge policy is not documentary omission. Unregistered absence paraphrases or additional assertions remain unresolved; speculative values fail. A decline on an answerable question fails.

## Metrics, repetitions and claim limits

Retain all forty planned IDs in each of three sessions per arm. Use one frozen permutation per paired repetition and the same order for both systems; declare the six-session order before execution. No new-bank tuning, model/prompt/schema/retrieval changes, regenerated answers or best-result retries after consumption. Infrastructure interruptions preserve original attempts and unfinished planned work; recovery never silently resubmits a question. Stop reason EOS and a ninety-second complete-query deadline are captured and audited separately. Warmup contains no documents or bank content.

Report complete supported-answer success, canonical factual correctness, emitted-citation precision and clean abstention separately, including lower/upper bounds for unresolved cases. Per-session qualification requires conservative lower bounds of 90% supported answers, 95% citations and 90% abstention, with thirty answerable/ten unknown questions. Errors, timeouts, missing/duplicate work and zero denominators cannot disappear. Installation smoke tests are not qualification.

The paired analysis retains all three repeats within each of forty unique question clusters. Seeded whole-question resampling is a fixed-corpus sensitivity envelope, not a p-value or population confidence claim. Do not count 120 repeats as independent questions/users. Latency includes retrieval, embedding and all model phases; parsing/import/index and cold start are separate. Native answer completion uses the unique documented final body-output event, separating later title generation. Capture uncertainty and unavailable timings stay explicit. No automatic market winner, privacy superiority, novice-installation or hardware-upgrade conclusion is inferred.