By Kingy AI · Reviewed by Curtis Pyke. Tested November 9 to 10, 2026.
Ask an AI chatbot what a passport costs, how much you can put in a retirement account this year, or what the minimum wage is, and it will usually answer in one confident sentence. We wanted to know how often that sentence is wrong.
So we asked nine AI models 125 everyday questions that each have one official answer, and marked every reply against the government or agency page that sets the figure. The full scorecard, every question we can publish and every reply are on the AI Accuracy Index. This is what we found.
The short version
- Working from memory, the best models gave a wrong figure about 1% of the time. The worst gave one about 20% of the time.
- GPT-6 Astra and GPT-6.1 Sol answered about 58% of questions correctly and were almost never wrong.
- Claude Opus 5.5 and Claude Sonnet 5.5 answered just as many correctly, but gave a wrong figure 8% to 10% of the time.
- Gemini 4 Argon and Grok 4.7 were almost never wrong, mostly because they declined to answer: they got only 12% and 19% right.
- With a working web search, nearly every answer was right.
How we tested
The 125 questions cover taxes, benefits, travel fees, wages, postage and product recalls in the US, UK, Canada, Australia, the EU, New Zealand and Ireland. Each one names the country, the year and the exact case, so there is only one right answer. Every answer key was written from the official page and then found again by a second checker who never saw the key.
Each model got the same prompt with the date, the question, and a request to finish with its answer or “NOT SURE”. We asked every question twice, with no web search, through each model’s developer access at default settings. Fixed rules, not another AI, decided whether each reply was right, wrong or not sure.
Being right is not the same as being safe
The most useful models were the ones that answered often and were rarely wrong. GPT-6 Astra gave a wrong figure on 0.8% of its answers and GPT-6.1 Sol on 1.2%. Both answered more than half the questions correctly.
The two Claude models answered a similar share correctly, but when they did not know, they were more likely to give an old number than to say so. Counted question by question, Claude Sonnet 5.5 gave a wrong figure on 14 questions where GPT-6 Astra did not, and there was no question where Astra was wrong and Sonnet was not.
There is one thing Claude did better. In 44 of its 47 wrong answers, it told the reader that it was going from memory and that they should check the official site. Only 2 of the 97 wrong answers from the other models carried any warning like that. If you read the whole reply, Claude usually tells you when not to trust it. If you only read the number, it does not help.
At the other end, Gemini 4 Argon never gave a wrong figure, and Grok 4.7 gave one once. They mostly said they were not sure, so they were rarely useful either. The open-weight Qwen 3.8 27B was wrong most often, at 19.6%, and often sent back no answer at all.
The usual mistake is last year’s number
Half of all wrong answers were exactly an earlier official figure. The models had learned the old fee or limit and did not know it had changed. Two examples:
- The US ESTA travel fee rose to $40.62 on October 16, 2026. Models answered $40.27, $40 or even $21, all of them figures that were right at some point.
- A US Forever stamp costs $0.82. Both Claude models said $0.78, the price before July 2026.
Travel fees were the riskiest area: 22% of answers about passports, visas and travel permits gave a wrong figure. Recalls were the opposite. Models said “not sure” to 94% of recall questions, which is the right call when they cannot know.
What to do about it
- Turn on web search for anything with a date or a price. When we gave every model the same search tool, 379 of 380 answers were right.
- Read the warnings. If a reply says it is going from memory, treat the number as a guess.
- Check the official page before you pay or file. A fee or limit that changed this year is exactly where models go wrong.
Limits and a conflict of interest
These are the models through their developer access, not the ChatGPT, Claude or Gemini apps, which add their own search and instructions. 125 questions is enough to show the large gaps above, not small differences such as Claude Opus against Claude Sonnet. We keep 39 of the questions private so the test cannot simply be learned.
Kingy AI’s research and writing are done with Claude, and Claude models are in this test. That is why the marking is done by fixed rules rather than by any AI, and why every public question, answer key, source and reply can be downloaded and checked.