# Kingy 1M Context Reality scoring rubric

Score each response against that case's `expected.json`. Do not infer a pass from
endpoint acceptance, a fluent answer, or a provider context claim.

## Metrics

- **Retrieval accuracy:** exact early/middle/end values plus the five distributed
  needles, divided by eight. Each value must be paired with its exact evidence ID.
- **Multi-hop accuracy:** 1 only when the vault code is exact and all three hop
  citations are present; otherwise 0.
- **Contradiction accuracy:** 1 only when the current percentage and winning
  evidence are exact and the obsolete passage is identified as superseded.
- **Code dependency accuracy:** 1 only when the final TTL is exact and all four
  active dependency citations are present.
- **Grounded synthesis accuracy:** 1 only when the recommended route is exact and
  all three independent constraints are cited.
- **Citation accuracy:** correct unique evidence IDs divided by all evidence IDs
  requested. Unknown or fabricated citations count against the numerator.
- **Hallucination rate:** 1 for the absent-fact task if the response supplies a
  value or supporting citation for the nonexistent identifier; otherwise 0.
- **Refusal/failure rate:** 1 when there is no parseable response, an API failure,
  or a refusal that prevents scoring; otherwise 0.

Report every trial. Aggregate with arithmetic means only after retaining the raw
trial rows. Keep cold and cache-probe latency separate. Never merge API rejection
with model accuracy: rejection is an integration/acceptance result.
