AI News

Why Do AI Rankings Disagree? Interactive Explainer

Skip to experiment

Explorable AI · 01

Same scores. Change the question.

One reason AI rankings disagree: they weight skills differently. Choose what matters for an editing or coding assistant and watch the winner change.

Illustrative calculation

Orbit, Muse and Bolt are fictional profiles—not real products or tested models. Orbit is balanced, Muse favors writing and Bolt favors coding. The frozen numbers isolate weighting; real rankings can also use different tasks, judges and evidence.

Start with a priority

Orbit leads with equal priorities.Fictional balanced profile · Orbit 82.33 · Bolt 78.33

What matters to you?

Give each skill 0–100 priority points. We turn their relative sizes into weights.

33.3% weight

Clear, useful prose

Ignore50 pointsPrioritize
33.3% weight

Solving programming tasks

Ignore50 pointsPrioritize
33.3% weight

Working through problems

Ignore50 pointsPrioritize

Keyboard: Tab to a slider, then use arrow keys. Zero ignores a skill; all zeros leaves no ranking.

Your rankingWeighted score / 100

Orbit takes the lead

Fictional balanced profile

Equal priorities

Orbit leads with equal priorities.

WritingCodingReasoning

With equal priorities, Orbit’s broad scorecard puts it first. Try “Writing first” to change the winner.

Nearest one-slider leader change

Hold the other two priorities fixed. Raise Coding from 50 to 93; Bolt takes first place. That is a 43-point move.

Equal-priority reference

01 Orbit82.3302 Bolt · 78.3303 Muse · 77.33

Only the priorities changed.

Share the restorable experiment or a labeled before-and-after image.

Look under the hood

Nothing here changed except the weights.

All three made-up models use the same 0–100 scale, with higher scores meaning better performance in an invented rubric.

Real benchmarks are not automatically comparable just because their scores share a scale.

The frozen, fictional scorecard
ModelWritingCodingReasoning
MuseFictional writing profile926476
BoltFictional coding profile689473
OrbitFictional balanced profile808087
Show the calculation for these priorities

Weighted score = sum of (priority × skill score) ÷ total priority points.

Bars show each skill’s contribution on a fixed 0–100 axis. Ranking uses exact integer totals; displayed scores are rounded.

The real-world connection

One mechanism. More than one reason.

This experiment isolates weighting. Real rankings can also differ because of their tasks, judges, prompts, model versions, missing coverage and statistical uncertainty.

It does not reproduce any real leaderboard or show that one is rigged. The priorities and linear averaging are teaching choices, not calibrated preferences.

Explorable AI 01 · Illustration v1 · Sources checked 5 September 2026