← Kingy Local AI

FULL THREE-PAIR DOCUMENT STUDY · WORKFLOW 1.5.4

Kingy and native LM Studio: three paired PDF repetitions

document-comparison-v19-20261015.1 · analysis completed 2026-10-17T23:17:24.253519+00:00

40 unique question clusters × 3 paired repetitions = 120 attempts per workflow.

Ten synthetic PDFs and 40 held-out questions were frozen before inference, then attempted in three original runs per configured workflow. Model bytes, generation seed, candidate and scoring were frozen. All 240 original attempts are preserved. Candidate 1.5.4 is held: all three Kingy runs missed the automatic quality targets, the independent source audit found concise-answer false failures in the scorer, and one native completion remains unverified after a host restart. Frozen automatic tallies are not a semantic accuracy ranking. The report documents source-verified differences in both directions. The active download remains workflow 1.4.0.

Each pair contains the same 30 answerable and 10 unknown questions. Repetitions measure repeatability on the declared system; these are 40 unique cases, not 120 independent questions or an independent physical reproduction.

Inference repair and independent checks

Each requested field now has separate retrieval, evidence selection and sufficiency checks, within one 90-second question deadline. The shared task-completeness scorer remains version 1.4.0. Consumed-corpus regression and independent synthetic controls preceded this separately authored fresh corpus. Finite participant-role checks can decline unfamiliar wording; they do not guarantee truth.

Separate-agent engineering and preserved scorer validation · Independent every-item corpus review · Field-isolated inference repair; shared scorer unchanged

Final independent source check and release decision

Candidate 1.5.4 is held because the final scoring-validity gate failed. Frozen scores do not establish qualification or comparative advantages.

All240 retained originals are covered by82 distinct answer/grade/citation bundles. Original output/source mapping and raw native text are preserved, with no identified parser loss or confirmed false pass. Semantic score validity does not pass: native contextual concise answers Q12/Q15 are definitively false-failed in all three repetitions. Many unresolved native paraphrases are semantically complete and must not be presented as wrong. Preserve every frozen score; issue no replacement leaderboard. Kingy has six repeated answerable declines and one conservative citation unresolved, and neither system establishes an overall winner. Workflow1.5 remains held independently for Kingy qualification and incomplete native original23.

The final audit did not pass the scoring-validity gate. No overall comparative quality winner is claimed. Unblinded AI-assisted independent source/answer review on one shared Mac, one synthetic corpus, one model weight set with two runtime/RAG pipelines. This judgment concerns semantic interpretation of frozen mechanical grades, not tampering or reproduction arithmetic. No regrading, alias repair, model calls, or numeric replacement accuracy was performed. Conservative unresolved cases remain unresolved in the published frozen record. The separate agent reviewed 82 distinct response bundles covering all 240 originals. This is an unblinded AI-assisted review, not human adjudication. The original scores and responses are preserved.

Final independent source and scoring audit · Every independently reviewed response bundle · Every Kingy attempt failed or unresolved under the frozen scorer · Actual launch decision and failed gates

Observed differences in both directions

These are observations on this fixed corpus and configuration, not a general product ranking.

Preserved frozen scores and effects — invalid for comparative claims

Frozen automatic scores, preserved

All A1/B1, A2/B2 and A3/B3 attempts are retained. Confirmed passes form the conservative rate. The upper bound additionally includes unresolved judgments. These bounds are not confidence intervals. Canonical facts and complete supported answers are reported separately. Displayed native retrieval cards count only when claimed in the answer; uniquely quote-mapped citations can receive credit, with page capability retained separately.

Frozen common rubric; all six complete sessions
Workflow / repetitionCanonical factsSupported answersCitation source supportCorrect abstentionAttempt accounting
Kingy workflow 1.5.4A124/30 (80.0%)0 unresolved · bounds 80.0%–80.0%23/30 (76.7%)1 unresolved · bounds 76.7%–80.0%27/28 (96.4%)1 unresolved · bounds 96.4%–100.0%10/10 (100.0%)0 unresolved · bounds 100.0%–100.0%40/40 retained0 errors · 0 timeouts
Native LM Studio PDF chatB17/30 (23.3%)18 unresolved · bounds 23.3%–83.3%7/30 (23.3%)15 unresolved · bounds 23.3%–73.3%31/35 (88.6%)0 unresolved · bounds 88.6%–88.6%10/10 (100.0%)0 unresolved · bounds 100.0%–100.0%40/40 retained0 errors · 0 timeouts
Kingy workflow 1.5.4A224/30 (80.0%)0 unresolved · bounds 80.0%–80.0%23/30 (76.7%)1 unresolved · bounds 76.7%–80.0%27/28 (96.4%)1 unresolved · bounds 96.4%–100.0%10/10 (100.0%)0 unresolved · bounds 100.0%–100.0%40/40 retained0 errors · 0 timeouts
Native LM Studio PDF chatB27/30 (23.3%)18 unresolved · bounds 23.3%–83.3%7/30 (23.3%)15 unresolved · bounds 23.3%–73.3%31/35 (88.6%)0 unresolved · bounds 88.6%–88.6%10/10 (100.0%)0 unresolved · bounds 100.0%–100.0%40/40 retained0 errors · 0 timeouts
Kingy workflow 1.5.4A324/30 (80.0%)0 unresolved · bounds 80.0%–80.0%23/30 (76.7%)1 unresolved · bounds 76.7%–80.0%27/28 (96.4%)1 unresolved · bounds 96.4%–100.0%10/10 (100.0%)0 unresolved · bounds 100.0%–100.0%40/40 retained0 errors · 0 timeouts
Native LM Studio PDF chatB36/30 (20.0%)18 unresolved · bounds 20.0%–80.0%6/30 (20.0%)15 unresolved · bounds 20.0%–70.0%30/34 (88.2%)0 unresolved · bounds 88.2%–88.2%9/10 (90.0%)0 unresolved · bounds 90.0%–90.0%40/40 retained2 errors · 1 timeouts

Errors and timeouts can overlap and are not added together. Both remain failed attempts. The frozen citation metric checks for a registered source anchor in the emitted excerpt. It does not independently guarantee that the answer states the right fact: a citation can contain the expected source anchor while the answer still names the wrong event or value. Whole-answer support requires factual correctness and source checks together. Citation denominators include emitted partial or unresolved citations. Atomic task-completeness rubric1.4.0 · Before-output comparison protocol

Qualification and unresolved judgments

The declared thresholds are at least 90% supported-answer success, 95% emitted-citation precision and 90% correct abstention in every repetition, using conservative lower bounds.

Unresolved semantic or citation judgments remain. Their possible outcomes remain in the displayed bounds; an unreviewed ambiguity does not establish a comparative winner. Qualification is an absolute threshold for this workflow and test scope. A relative score does not automatically qualify a buying recommendation.

Paired effects and uncertainty

These effect calculations did not pass the scoring-validity gate. They are retained for audit and must not be used as accuracy, citation-quality or abstention superiority claims.

Effects below are Kingy minus native LM Studio. The 10,000-draw sensitivity analysis resamples whole unique-question clusters and keeps the three repetitions together. The envelope describes case-resampling sensitivity on this fixed corpus. It is not a population confidence interval, p-value or market-wide ranking.

Arithmetic unresolved bounds and whole-question sensitivity
Outcome / clustersEffect bounds95% case-resampling sensitivity envelope
complete question success40 unique clusters4.2 pp to 44.2 pp-10.8 pp to 60.8 pp
supported answer success30 unique clusters4.4 pp to 57.8 pp-15.6 pp to 76.7 pp
canonical fact correctness30 unique clusters-2.2 pp to 57.8 pp-20.0 pp to 76.7 pp
correct abstention10 unique clusters3.3 pp to 3.3 pp0.0 pp to 10.0 pp
emitted citation precision40 unique clusters8.0 pp to 11.5 pp-4.9 pp to 23.8 pp

No automatic winner is declared. Paired calculations and unresolved bounds

Complete-answer latency

B1 continued after original 35 with a verified replacement blank template, a new model load and neutral warmup. All 40 planned originals remain retained. Unscheduled preparation activity and the restart prevent an uninterrupted-session timing claim.

Descriptive time includes each workflow's full answer pipeline. Native timing includes native preprocessing, embedding/retrieval and inference. Kingy timing includes retrieval, selection, source resolution and sufficiency checks. Import, cold start and neutral warmup are recorded separately. Median and nearest-rank p95 include retained measured failures; unavailable observations remain visible. These measurements do not establish model throughput, novice setup speed or a population speed advantage.

Seconds; descriptive observations only
SessionMeasured / unavailableMedianp95Maximum
A140 / 05.9489.80813.893
A240 / 03.6557.4237.788
A340 / 03.6697.4727.790
B140 / 012.86520.87340.455
B240 / 06.6788.4688.855
B339 / 17.57918.60192.536
Actual timing boundaries

Control provenance and instrumentation amendment

A bounded Duplicate-save acknowledgement wait was added during native B1, after 20 original responses. The candidate, corpus, model, factual prompts, settings, common scorer and answer timing boundaries remained unchanged. Earlier originals and control versions are retained. Control interruptions do not establish native app reliability.

Native LM Studio used the separately reviewed direct System Events template-menu control. After two blank-copy persistence delays, an independently reviewed wrapper was added before B1 original21. It waits up to ten seconds for a single acknowledged Duplicate action to persist a blank ten-attachment record, then returns to unchanged frozen checks. It adds no UI retry, model request, question submission or change to Send/answer timing. After original35, two unscheduled preparation generations occurred outside every measured answer window. The first followed unexpected short input; the user later reported concurrent typing, but exact event attribution was not established. The second contained file parts without text during menu preparation; its cause remains unresolved. Both affected native chats and their original bytes were preserved privately. Coordinate-based measured-trial template-menu preparation was retired before original36. An independently reviewed PID-scoped AXPress control verifies the exact owned row, outer AXGroup popup trigger, enabled popup state and labelled inner More options child, then presses the outer trigger once. It requires a focused More menu, and the binding rejects any second menu, including unnamed native overlays. An initial inner-button target mismatch and stale dual-menu guard defect were corrected before original36, with prior sources and failed probes preserved. The installed vendor UI source was inspected read-only; vendor code was not changed. The blank ten-attachment template is checked before and after preparation. The neutral native preflight created zero-file unscored copies once each and proved canonical reselection and repeat menu access without inference. Foreground indicators and Cancel acknowledgments were not treated as proof of selection or dismissal. A fresh owned blank chat was renamed and five PDFs were sequentially attached with exact PDF/configuration checks. The sixth sequential upload was visibly refused by the installed default five-file add handler. No policy, limit or vendor code was changed, and no over-limit upload was forced. Independent review allowed conditional reuse of a pre-existing, never-measured blank already containing all ten frozen PDFs, through ordinary native identity, name-only and duplicate controls. The native arm therefore depends on an existing ten-file conversation state; this does not establish that a fresh sequential ten-file setup works or that the UI has a universal five-file limit. Native Reveal in Finder proved the exact unused ten-PDF record before it was renamed. The five-file preparation was preserved under a quarantine name, and only the verified ten-file blank name changed to the canonical template name. One ordinary native Duplicate then produced a blank ten-file copy with exact frozen hashes and ten visible previews; the required screenshot API passed. The final model selection and eleven configuration keys are bound by the later AX freeze and verification receipts. Later draft-only characters were preserved privately; the user reported concurrent Mac use, but exact attribution remains unresolved. After the user confirmed an idle Mac, a clean unloaded GUI restart restored fresh observation. New inference was held while a separate finite local-model job occupied the shared host. The study model was kept unloaded during UI recovery. B1 used three recorded model loads and three neutral warmups in total, including two extra cold loads and two neutral READY warmups after original35. Each warmup was one model request without documents or a bank question. All receipts are retained. During attachment-only preparation, another job loaded a separate model and selected runtime2.60 while frozen2.56 remained installed. All34 frozen app, model, RAG and runtime artifacts were reverified unchanged. That observation does not identify who changed the selection or prove a cause for UI interruptions. A mistaken sheet-count diagnostic was corrected against an explicit native sheet lookup. Normal window and sidebar layout changes restored full-width controls; a fresh native menu-item point click opened the actual picker. These preparation actions did not change candidate inference, grading or retained original attempts. Extra conditioning and preparation pauses can affect cache state. Native runs include preparation pauses; B1 also includes extra cold loads and is not an uninterrupted-session timing result. Four unrelated input/output log events are omitted from the public derivative with exact hashes disclosed; all other event bytes and original private logs are preserved. The public engine-process proof omits a local credential-bearing command and binds the unchanged private original by hash. Independent export checks verify the derivative and raw-receipt exclusion; the final completed public artifacts still require the actual privacy scan. Before B3 original1, the native control connection closed before a response. Reconciliation proved zero retained or pending questions, an unconsumed menu token and only the two neutral warmup log events. A documented control reset recovered the actual singleton menu; the old token and unchanged log prefix were preserved. No model reload, extra warmup, control-source edit or question replay was needed. Before B3 original2, the attachment guard rejected one newly duplicated blank: it had ten entries but only nine distinct backing-file identifiers and hashes. The canonical source still had all ten correct frozen PDFs. The rejected blank was preserved unchanged and its native identity verified. A fresh ordinary copy passed ten distinct frozen PDF hashes before any input; original2 was then submitted once. The same guard also rejected a second blank before original22, before prompt, pending record or Send. Its separate interruption and recovery receipts preserve that preparation event. These storage-identifier observations do not establish which filenames the UI or model displayed, their cause, or general product reliability. B3 scheduled original23 (V19-Q09) was submitted before an unobserved gap spanning a host restart. The saved record has no documented final answer, final-generation block or matching final model-log event. Old owned processes were unavailable; a new main app instance existed and remained uncontrolled during reconciliation. The exact native record, pending clock and46-event log prefix were preserved. An initial capture-helper preflight required an explicit null timestamp while the original JSON omitted that field; it stopped before a marker, journal write or coordinator call. A separately reviewed correction accepts only absence or explicit null and rejects every present completion value. The approved capture-only helper then invoked the unchanged frozen coordinator once, using a truthful observation scoped to the unavailable old owned instance and forbidding all UI/model actions. It retained original23 as incomplete without inventing UI completion, answer timing or an answer, and cleared the pending slot. No question was resubmitted. The missing final evidence does not prove that no model request occurred, a wrong model answer, or product unreliability. A fresh neutral ten-PDF duplicate, screenshot-API, canonical reselection and single focused menu preflight then passed on the restarted main instance. A separate reviewer bound the exact frozen2.56 engine, all eleven configuration keys, effective SDK settings, unchanged ten PDF hashes,48 log lines and original46-event prefix before allowing original24 onward. One additional cold model load and one neutral warmup followed the host restart. Their actual continuation and final restoration are disclosed in the linked final load/warmup inventory. No comparative advantage may be claimed from this interrupted row. After that preflight, a focus-only helper observed native focus, but CUA omitted its focused-element metadata. That pilot did not pass and was stopped; unavailable metadata does not prove native focus was absent. A distinct transport amendment moved the current native focus check to the same bounded call as exactly one Duplicate AXPress. Its source, failed neutral attempts and actual verification are preserved separately. This changes control transport only; frozen candidate, scorer, corpus, later prompt and Send behavior remain unchanged. Kingy A2 initially stopped before any server load, warmup or question submission because a model link pointed to a disconnected external volume. The zero-attempt start was preserved. The link was repaired to the existing local native model after exact frozen-byte verification and a separate metadata-only review. Candidate, corpus, scorer and model bytes did not change, and no measured question was replayed. All planned original submissions are retained. Preparation interruptions and unscheduled activity are not product reliability evidence.

Focus-only pilot stopped: CUA metadata did not establish the required boundary · Native focus observation and unavailable CUA focus metadata · Distinct source pins for actual native Duplicate boundary · Separate-agent source review for the finite native Duplicate boundary · Actual same single blank copy, ten PDF hashes, settings and visible previews · Independent actual neutral-copy binding with late acknowledgment disclosed · Actual resumed original24 model and five generation/configuration groups match the frozen baseline · Native transport frozen before fresh corpus · Independent native transport review · Bounded preparation wait added after B1 original20 · Independent bounded-wait review and controls · Independent original-attempt integrity and template recovery review · Independent replacement-template validation · Native UI preservation, PDF and configuration checks · AXPress-only native menu and replacement-template pins · Independent AXPress guard and native preflight review · Historical zero-file native menu preflight without inference · Actual ten-PDF duplicate, canonical reselection and single focused menu · Actual selected model, ten PDFs and eleven frozen configuration fields · Actual frozen runtime2.56; credential-bearing command omitted · Previous and frozen runtime selection receipts · Actual native loads, neutral warmups and final runtime-selection restoration · Zero-question native control-connection interruption · Unconsumed token and zero-inference native connection recovery · Rejected blank copy with duplicate backing-file identifiers before input · Failed blank preserved; ten distinct frozen PDF hashes verified before original2 · Second rejected blank copy before original22 · Original22 preparation recovery with failed blank preserved · Interrupted pending native23: preserved record and log observations · Current boot occurred after the interrupted native record · Old owned processes unavailable; new main instance observed without control · All34 frozen runtime, model, app and RAG artifacts still match after restart · Independent preserved-original and capture-only continuation review · Initial capture-helper review; preflight later refused an absent timestamp field · Initial capture preflight stopped before any action or journal write · Exact revised capture helper and absent-or-null guard independently approved · Interrupted native23 retained once with zero UI/model actions and no completion claim · Root verifies exact interrupted original, cleared pending slot and zero resubmissions · Fresh post-restart neutral ten-PDF duplicate and actual frozen runtime/settings · Independent binding of fresh preflight and conditional continuation without retries · Independent outer popup trigger and overlay guard review · Actual single focused menu from one neutral outer AXPress · Installed native UI target diagnosis; vendor code unchanged · Read-only installed attachment callback diagnosis · Observed refusal of a sixth sequential native PDF upload · Read-only installed default file-count guard diagnosis · Independent conditional reuse review of an unused10-PDF native blank · Native Reveal proves the exact unused ten-PDF blank identity · Only the verified ten-PDF blank name changed · Actual ten-PDF blank duplicate and screenshot API preflight · Actual layout recovery and corrected sheet-count diagnostic · Unrelated model and changed runtime selection observed during preparation · All34 frozen app, model, RAG and runtime artifacts unchanged · Public log derivative: four unplanned events omitted, originals preserved · User-reported concurrent use during preparation · Later draft-only activity: user-reported concurrent Mac use · Separate local job observed; new measurements held · Independent local-credential export fix and thirteen focused checks · Independent private neutral screenshot exclusion and five scanner cases · Zero-question Kingy startup failure and byte-identical model-link recovery · Existing local model matches the frozen GGUF bytes · Independent zero-inference and unchanged-input recovery check · Preserved zero-attempt Kingy startup failure · Originals sealed before reference access

Exact configurations and failures

Kingy settings
{
  "workflowVersion": "1.5.4",
  "runnerSha256": "911298ba21797fe0704874ed85c8f306193de2a69d130ea7ebb32b18bb9064f7",
  "configuration": {
    "machine": {
      "platform": "apple-silicon",
      "chip": "M4 Pro",
      "model": "Mac16,7",
      "cpuCores": 14,
      "gpuCores": 20,
      "memoryGiB": 24,
      "osFamily": "macOS",
      "osVersion": "26.3",
      "osBuild": "25D125"
    },
    "runtime": {
      "name": "llama.cpp",
      "version": "b11429",
      "commit": "d81235049",
      "binarySha256": "51afbf4ea6485b9ada19e99af97181f12baca42ef704b1e291c26a9ac0644eae"
    },
    "python": "3.12.14",
    "pypdf": "6.10.0",
    "model": {
      "id": "Qwen3-4B",
      "quantization": "Q4_K_M",
      "repository": "Qwen/Qwen3-4B-GGUF",
      "revision": "bc640142c66e1fdd12af0bd68f40445458f3869b",
      "file": "Qwen3-4B-Q4_K_M.gguf",
      "bytes": 2497280256,
      "sha256": "7485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5",
      "license": "Apache-2.0"
    }
  },
  "generationSeed": 271828,
  "context": 8192,
  "temperature": 0,
  "thinking": false,
  "cpuThreads": 10,
  "gpuOffload": "full Metal",
  "parallel": 1,
  "flashAttention": true,
  "kv": "F16",
  "selectionOutputLimit": 256,
  "sufficiencyOutputLimit": 192,
  "answerModelRequests": [
    90,
    90,
    90
  ],
  "warmupRequests": 3,
  "cachePolicy": "cache_prompt false; no claim that OS file caches were flushed.",
  "timingBoundary": "runner.ask entry through its complete returned answer, source resolution and sufficiency check; excludes prior PDF import/index, cold start, warmup and later grading.",
  "nativeEgressComparison": "unverified; no comparative privacy claim"
}

Actual Kingy inference configuration

Native LM Studio settings
{
  "load_config": {
    "fields": [
      {
        "key": "llm.load.llama.autoFit",
        "value": false
      },
      {
        "key": "llm.load.mlx.autoFit",
        "value": false
      },
      {
        "key": "llm.load.llama.cpuThreadPoolSize",
        "value": 10
      },
      {
        "key": "llm.load.contextLength",
        "value": 8192
      },
      {
        "key": "llm.load.llama.flashAttention",
        "value": true
      },
      {
        "key": "llm.load.promptTemplate",
        "value": {
          "type": "jinja",
          "jinjaPromptTemplate": {
            "template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for index in range(ns.last_query_index, -1, -1) %}\n    {%- set message = messages[index] %}\n    {%- if ns.multi_step_tool and message.role == \"user\" and not('<tool_response>' in message.content and '</tool_response>' in message.content) %}\n        {%- set ns.multi_step_tool = false %}\n        {%- set ns.last_query_index = index %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set content = message.content %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in message.content %}\n                {%- set content = message.content.split('</think>')[-1].lstrip('\\n') %}\n                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- endif %}\n{%- endif %}"
          }
        }
      },
      {
        "key": "llm.load.llama.tryDirectIO",
        "value": false
      },
      {
        "key": "llm.load.llama.acceleration.offloadRatio",
        "value": 1
      },
      {
        "key": "llm.load.numParallelSessions",
        "value": 1
      },
      {
        "key": "llm.load.seed",
        "value": {
          "checked": true,
          "value": 271828
        }
      },
      {
        "key": "llm.load.yuzu.autoFit",
        "value": false
      },
      {
        "key": "llm.load.vllm.autoFit",
        "value": false
      }
    ]
  },
  "prediction_config": {
    "fields": [
      {
        "key": "llm.prediction.promptTemplate",
        "value": {
          "type": "jinja",
          "jinjaPromptTemplate": {
            "template": "{%- if tools %}\n    {{- '<|im_start|>system\\n' }}\n    {%- if messages[0].role == 'system' %}\n        {{- messages[0].content + '\\n\\n' }}\n    {%- endif %}\n    {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n    {%- for tool in tools %}\n        {{- \"\\n\" }}\n        {{- tool | tojson }}\n    {%- endfor %}\n    {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n    {%- if messages[0].role == 'system' %}\n        {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n    {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for index in range(ns.last_query_index, -1, -1) %}\n    {%- set message = messages[index] %}\n    {%- if ns.multi_step_tool and message.role == \"user\" and not('<tool_response>' in message.content and '</tool_response>' in message.content) %}\n        {%- set ns.multi_step_tool = false %}\n        {%- set ns.last_query_index = index %}\n    {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n    {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n        {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n    {%- elif message.role == \"assistant\" %}\n        {%- set content = message.content %}\n        {%- set reasoning_content = '' %}\n        {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n            {%- set reasoning_content = message.reasoning_content %}\n        {%- else %}\n            {%- if '</think>' in message.content %}\n                {%- set content = message.content.split('</think>')[-1].lstrip('\\n') %}\n                {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n            {%- endif %}\n        {%- endif %}\n        {%- if loop.index0 > ns.last_query_index %}\n            {%- if loop.last or (not loop.last and reasoning_content) %}\n                {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n            {%- else %}\n                {{- '<|im_start|>' + message.role + '\\n' + content }}\n            {%- endif %}\n        {%- else %}\n            {{- '<|im_start|>' + message.role + '\\n' + content }}\n        {%- endif %}\n        {%- if message.tool_calls %}\n            {%- for tool_call in message.tool_calls %}\n                {%- if (loop.first and content) or (not loop.first) %}\n                    {{- '\\n' }}\n                {%- endif %}\n                {%- if tool_call.function %}\n                    {%- set tool_call = tool_call.function %}\n                {%- endif %}\n                {{- '<tool_call>\\n{\"name\": \"' }}\n                {{- tool_call.name }}\n                {{- '\", \"arguments\": ' }}\n                {%- if tool_call.arguments is string %}\n                    {{- tool_call.arguments }}\n                {%- else %}\n                    {{- tool_call.arguments | tojson }}\n                {%- endif %}\n                {{- '}\\n</tool_call>' }}\n            {%- endfor %}\n        {%- endif %}\n        {{- '<|im_end|>\\n' }}\n    {%- elif message.role == \"tool\" %}\n        {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n            {{- '<|im_start|>user' }}\n        {%- endif %}\n        {{- '\\n<tool_response>\\n' }}\n        {{- message.content }}\n        {{- '\\n</tool_response>' }}\n        {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n            {{- '<|im_end|>\\n' }}\n        {%- endif %}\n    {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n    {{- '<|im_start|>assistant\\n' }}\n    {%- if enable_thinking is defined and enable_thinking is false %}\n        {{- '<think>\\n\\n</think>\\n\\n' }}\n    {%- endif %}\n{%- endif %}"
          },
          "stopStrings": []
        }
      },
      {
        "key": "llm.prediction.llama.cpuThreads",
        "value": 10
      },
      {
        "key": "llm.prediction.repeatPenalty",
        "value": {
          "checked": true,
          "value": 1
        }
      },
      {
        "key": "llm.prediction.topPSampling",
        "value": {
          "checked": true,
          "value": 1
        }
      },
      {
        "key": "llm.prediction.minPSampling",
        "value": {
          "checked": true,
          "value": 0
        }
      },
      {
        "key": "llm.prediction.temperature",
        "value": 0
      },
      {
        "key": "llm.prediction.maxPredictedTokens",
        "value": {
          "checked": true,
          "value": 512
        }
      },
      {
        "key": "llm.prediction.reasoning.enableThinking",
        "value": false
      },
      {
        "key": "llm.prediction.contextOverflowPolicy",
        "value": "stopAtLimit"
      },
      {
        "key": "llm.prediction.tools",
        "value": {
          "type": "none"
        }
      }
    ]
  },
  "plugins": [
    "lmstudio/rag-v1"
  ],
  "plugin_configs": {
    "lmstudio/rag-v1": {
      "preset": "",
      "config": {
        "fields": [
          {
            "key": "retrievalLimit",
            "value": 8
          }
        ]
      }
    }
  },
  "system_prompt": ""
}

Effective native settings checked against originals

Engine and embedding artifact pins · Unmodified official native RAG artifact pins · Complete method, candidate and corpus freeze · All six original-session integrity checks

All120 Kingy originals with frozen grades · All120 native LM Studio originals with frozen grades

Limits and history

The original immutable v1.4 paired pilot remains separate historical evidence, including its incomplete native repetitions and post-output source review. Its scores and files are preserved. The original failed 1.5.0 study and its scorer limitation also remain unchanged. The original failed 1.5.1 study also remains unchanged. This repaired 1.5.4 report uses its own frozen rubric and fresh study; it does not rewrite earlier comparisons. Original immutable1.4 manifest · Original1.5.0 manifest

Inspect and reproduce

Report data · File hashes and sizes · Analysis completion and output hashes