The same tasks. The complete outputs. Dated evidence.
Choose an AI tool using the work it produced
These comparisons test specified API models on the same practical tasks. Every scored attempt will remain visible, including mistakes and failures. Coding and applied math are the first categories.
Python coding tasks
12 tasks · 6 dated configurations · 3 repetitions
Last tested: 2026-10-07T23:20:49.408221+00:00
Invoice reconciliation, CSV parsing, deployment order, rate limits, Unicode handling and other function tasks.
Applied math tasks
12 tasks · 6 dated configurations · 3 repetitions
Last tested: 2026-10-07T23:26:26.061111+00:00
Cash flow, probability, production constraints, loan payments, margins and statistics.
What has been tested
648 provider requests are recorded in the benchmark cohorts.
How updates work
Official sources and page health are checked on the host every 24 hours. Selected new configurations are tested once after review, within a US$20 monthly ceiling for expansion costs. Historical baseline costs remain under their existing allowances. Paid monthly full-roster reruns are disabled. Eligible launches target publication within 72 hours; older catch-up runs are labelled backfills. A source check does not change a test result. A run must fit its approved spending allowance.
Launch coverage
- Qwen3.8-Omni-Flash · announced 2026-09-18 · Pending provider access and contract; no test yet
- Grok 4.7 · announced 2026-09-21 · Pending provider access and contract; no test yet
- GPT-6 Sol · announced 2026-09-22 · Full Standard run exceeds the monthly expansion allowance
- GPT-6 Luna · announced 2026-09-22 · Tested on the v2 bank
- Claude Opus 5.5 · announced 2026-09-22 · Tested on the v2 bank
- Reflection Beam · announced 2026-10-05 · Restricted access or unverified pricing; no test yet
- Mistral Large 4 · announced 2026-10-06 · Pending provider access and contract; no test yet
- GLM-5.3-Flash · announced 2026-10-03 · Pending provider access and contract; no test yet
gpt-6-luna · Backfill · original deadline 2026-09-25T00:00:00+00:00 to 2026-09-26T00:00:00+00:00 (official API availability date only) · completed
claude-opus-5-5 · Backfill · original deadline 2026-09-25T00:00:00+00:00 to 2026-09-26T00:00:00+00:00 (official API availability date only) · completed
claude-haiku-5-5 · Launch target · original deadline 2026-10-10T00:00:00+00:00 to 2026-10-11T00:00:00+00:00 (official API availability date only) · completed
Human preferences
Writing style, logo aesthetics and visual editing quality will use separate blind audience votes. Technical checks cannot establish taste. No language model scores the quality of another model’s output. Inspect the audience gallery status.
The Kingy Brief
Get The Kingy Brief.
AI changes, original tests and one practical thing to try. Fridays at 09:00 Vancouver time.
Free · Double opt-in · Unsubscribe anytime
Signup help and newsletter schedule
Signup form provided by Beehiiv. After submitting, check your inbox for "Confirm your subscription to The Kingy Brief" and open its confirmation link. Check Spam or Promotions if you cannot find it.
Fridays at 09:00 Vancouver time: source-checked AI changes, original tests and one practical thing to try. The weekly restart begins October 9, 2026. We skip a week when there is not enough verified material. Free. Unsubscribe anytime.