AI News

Gemini 4 Neon hit the token cap in all three calls vs GPT-6.2 Astra

Gemini 4 Neon hit the 10,000-token completion cap on all three calls in Kingy.ai’s scanned-document test and returned no answers. GPT-6.2 Astra scored 52 of 54. Each Neon call reported 9,997 reasoning tokens, and all 54 questions went unanswered. A higher cap remains untested.

Kingy.ai paid for both tests. Google and OpenAI did not sponsor them.

We ran the follow-up on November 21, 2026 Pacific time, using three synthetic vendor-audit packets with conflicting amendments, lookalike identifiers and scanned registers. A correct answer needed both the expected value and the complete, exact set of record citations. In an earlier text-only pilot, both models scored 48 of 48.

Scanned-register follow-up Gemini 4 Neon GPT-6.2 Astra
Value and citations both correct 0 / 54 52 / 54
Correct values, separately 0 / 54 52 / 54
Correct citation sets, separately 0 / 54 52 / 54
Median request time 82.354 seconds 20.913 seconds
Total charge for three calls $0.786483 $1.421102

Kingy.ai’s November 21 follow-up. All 54 Neon questions went unanswered. Times are end-to-end request times and charges come from OpenRouter’s billing records.

Both models scored 48/48 on text. On scans, Neon hit the cap on all three calls and gave no answers (0/54). Astra scored 52/54.

What the harder test asked

Each model received the same three related packets, one fresh call per packet. Each packet contained about 24,000 words of text, including the synthetic archive, six active work orders and 18 questions. It also included three scanned tables. The tables held billed rates, receiving counts and posted adjustments. The scans had mild skew, blur and copy speckle; a blank rate was deliberately missing. The models had to match identifiers across documents, apply the controlling amendment and calculate an invoice or hold it when the evidence required that.

The primary score awarded a point only when the answer matched the prepared value and the complete set of requested citation IDs. Missing, extra or duplicated citations lost the point. Citations could appear in any order. The table also shows how often each model met either condition on its own. Astra scored 16/18, 18/18 and 18/18 across the three packets. Each Neon response had null answer content and scored 0/18.

Astra’s two misses came from one lookalike receiving row in packet 1. It selected row R1-511, for intake I1-104O and lot BO5, instead of R1-996, for I1-1040 and B05. It reported 151 units and a $4,778.71 invoice. The correct row yielded 162 units and $5,122.02. Its arithmetic on the inputs it selected was internally consistent, but the row selection carried through to the invoice.

Across the 18 questions in each category, Astra scored 18/18 on rate and amendment precedence, 17/18 on quantity and scanned records, and 17/18 on invoices or holds. Astra returned the expected hold code and citations for all six held invoices. Neon supplied no invoice amount or hold decision.

Settings, cost and time

We requested High reasoning effort for both models through OpenRouter, with provider fallback disabled and a 10,000-token cap per request for reasoning and visible output combined. The same text and three original PNGs were sent to both models, without OCR, retrieval, tools or follow-up prompts. Neon used Google Vertex global and returned snapshot google/gemini-4-neon-20261118; Astra used OpenAI and returned openai/gpt-6.2-astra-20261117. Compute and image preprocessing can vary by provider at the same effort setting. OpenRouter’s image-input documentation explains how images are submitted.

OpenRouter charged $2.207585 for the six requests. Astra’s total included a $0.244902 cache-write premium and zero recorded cache hits. Neon incurred charges on all three requests even though each came back without an answer.

The earlier text-only pilot

On November 18, 2026 Pacific, each model answered all 48 questions correctly across three synthetic facilities casebooks. The tasks involved joining records, applying corrections, calculating invoices and citing exact record sets. Both scored 16/16 on each packet. The pilot used text only, with roughly 32,000 input tokens per Neon call and 29,000 per Astra call; model token counts differ.

Neon’s three pilot calls cost $0.567945 and had a median request time of 52.941 seconds. Astra’s three calls had a median of 9.539 seconds and cost $0.958982, including about $0.18 in cache-write charges with zero recorded cache hits. These were different requests from the scan follow-up, so their costs and timings should be read separately. Two earlier pilot requests were rejected before inference and produced no answers. We retained the first completed response for each model and packet in both tests.

Sources and benchmark evidence

The downloadable benchmark evidence ZIP contains the test materials, visible responses, scoring code and billing fields extracted from the saved OpenRouter records.

Receipt2026-11-21
Tested
Three synthetic scanned packets, one call per model per packet, 54 questions per model; earlier text-only pilot: three packets and 48 questions per model.
Cloud side
OpenRouter: Gemini 4 Neon via Google Vertex global; GPT-6.2 Astra via OpenAI. Snapshots: google/gemini-4-neon-20261118 and openai/gpt-6.2-astra-20261117.
Settings
High reasoning requested for both; provider fallback disabled; 10,000-token cap per request for reasoning and visible output combined. No OCR, retrieval, tools or follow-up prompts.
Grading
A point required the expected value and complete exact citation set; missing, extra or duplicated citations lost the point. Citation order did not matter.
Cost
Scanned follow-up six calls: $2.207585 total (Neon $0.786483, Astra $1.421102). Text-only pilot: Neon $0.567945, Astra $0.958982.
Earlier pilot date
November 18, 2026 Pacific.
Scanned results
Neon reached the 10,000-token cap on all three calls and gave no answers (0/54). Astra scored 52/54.
Funding
Kingy.ai paid for both tests. Google and OpenAI did not sponsor them.
Dates
run 2026-11-21 · checked 2026-11-21

Limits

Each test used three related synthetic packets and one request per model per packet. That gives a narrow view of accuracy, latency and cost. The follow-up tested three images per packet with roughly 37,000 to 41,000 input tokens per request. Search, larger scan collections and repeat runs remain untested. Provider routes, settings and preprocessing can affect the result. For consequential work, check the cited rows and calculations against the original documents.