Kingy Sonnet vs Opus test kit

Use this to compare Claude models on a small job with a known right answer.

1. Open two fresh chats, one on each model, at the effort setting you normally use.
2. Paste the prompt below into both chats. Repeat at least twice per model.
3. Compare the answers with the answer key. Then swap in a real task of your own and write its pass conditions first.

PROMPT

Do not use tools, files, browsing, or external information. Return the requested JSON only.

You are checking synthetic supplier quotes for a four-seat team buying 12 months of service. Use only these rows. Q1: A costs USD18 per seat per month, paid monthly, plus a one-time USD40 setup fee. Q2: B costs USD16 per seat per month, billed for all 12 months upfront, plus a one-time USD100 setup fee. Q3: C costs EUR15 per seat per month, no setup fee; no EUR/USD exchange rate is supplied. Ignore tax for every quote. Total first-year budget is USD900; upfront cash is not separately constrained. Return only a JSON object with eligible_totals_usd (object keyed A and B), recommendation (supplier letter), excluded (object keyed by supplier with the missing information), and source_rows (object mapping A/B/C to the source row IDs). Calculate first-year totals, recommend the cheapest qualifying USD quote, and do not invent currency conversion.

ANSWER KEY

A = 4 x 18 x 12 + 40 = 904. This is over budget.
B = 4 x 16 x 12 + 100 = 868. This is the correct choice, with 32 left.
C stays unknown because no exchange rate is supplied.
The answer must be plain JSON, with no Markdown fences.

OPTIONAL SCORER

Save your answers in a text file with a line holding only --- between them, then run:
python3 check_quote.py your-answers.txt

kingy-oct-19-outputs.txt holds Kingy's October 19, 2026 outputs, Opus 5.5 twice then Sonnet 5.5 twice, both at Medium effort in the Claude web app. This is one small synthetic test, not a general accuracy score.
