OpenAI says its GPT-6 Astra model earned an average 66.7% partial-credit score on BioBasicsBench, a new assessment built around interpreting wet-lab evidence. That is the highest reported mean among the seven models tested at the same xhigh reasoning setting. It is also a long way from evidence that an AI assistant can be trusted to make an important laboratory decision on its own.
The benchmark announcement, published November 6, 2026, describes 100 tasks developed with Ligo, with expert-written scoring rubrics and repeated model attempts. The inputs include images, video and instrument files. The result is useful because it asks models to interpret the kinds of evidence scientists encounter, rather than only answer questions about biology. OpenAI reports the results; Kingy has not reproduced them. Not tested by Kingy.
For AI readers, the benchmark is a way to examine a model’s interpretation of scientific evidence. For lab teams, it is a starting point for deciding what to test locally. Neither group should read the headline score as a rate of successful experiments or correct lab decisions.
What BioBasicsBench measures
OpenAI says it built BioBasicsBench with Ligo around 100 wet-lab tasks spanning five domains: proteins, imaging and cytometry, assays and quantification, nucleic acids, and cloning. The material includes images, videos and instrument files, so an answer may depend on extracting information from evidence as well as applying biological knowledge. The report describes expert rubrics, review by another Ligo scientist and review by an external wet-lab PhD. Source: OpenAI
Those review steps concern the construction and assessment of the tasks. They do not establish that an outside group independently reproduced the model scores. Three separate questions matter: were the tasks and rubrics reviewed by relevant experts, how did the vendor’s model runs perform, and can another team reproduce those results? The announcement provides evidence for the first two; Kingy has not found evidence of the third in its bounded source check.
The tasks cover several kinds of evidence, but 100 tasks cannot represent every instrument, sample, experimental context or decision a laboratory might face. The overall score also does not tell a reader which domains or individual error types drove a model’s result.
The seven xhigh results
OpenAI reports results across seven models and 37 model-and-effort settings. The table below uses the seven rows with the same xhigh setting; the report also presents max settings separately. The score is mean rubric reward, including partial credit. Output tokens per task describe generated output, not total compute or a bill.
| Model | Mean reward | 95% confidence interval | Output tokens per task |
|---|---|---|---|
| GPT-6 Astra | 66.7% | 60.3–73.1% | 4,704 |
| GPT-6.1 Sol | 62.0% | 55.6–68.4% | 5,238 |
| GPT-5.6 Sol | 43.8% | 37.6–50.0% | 9,099 |
| GPT-6 Luna | 38.3% | 32.3–44.5% | 13,120 |
| GPT-5.5 | 35.4% | 29.7–41.0% | 11,135 |
| GPT-5.4 | 33.2% | 28.3–38.4% | 13,717 |
| GPT-5.2 | 31.2% | 26.2–36.3% | 27,620 |
Chart: OpenAI-reported mean partial-credit reward at xhigh reasoning effort. The table gives 95% confidence intervals. Source: OpenAI.
On the displayed rounded figures, Astra’s 66.7% mean is 22.9 percentage points above GPT-5.6 Sol’s 43.8%. As a relative change in the reported mean, that is approximately 52.3%. It does not mean Astra completed 52.3% more tasks. Astra’s reported output was about 48.3% fewer tokens per task than GPT-5.6 Sol’s; that figure alone says nothing definitive about runtime, total cost, energy or overall compute.
Astra’s mean is 4.7 percentage points above GPT-6.1 Sol’s. Their individual confidence intervals overlap, but overlap alone does not settle whether a paired difference is statistically significant. A proper comparison would need a suitable interval for the difference, ideally using the same task-level observations. The inspected announcement does not provide the task-level data needed for Kingy to calculate one.
What partial credit means
A mean reward of 66.7% does not say that the model solved exactly 66.7 of the 100 tasks. A task can award credit for some parts of an answer even when another part is wrong.
Here is a hypothetical explanation, not a description of the benchmark’s actual rubric weights: suppose a task has four equally weighted criteria and an answer satisfies three. It receives 75% credit. If the missing criterion is the one a scientist needs for the next decision, the answer may still be unsuitable for that purpose. A useful evaluation needs to inspect the mistakes as well as the average.
Two models could also have the same average while failing in different ways. A model that asks for clarification in an ambiguous case may be more useful than one that gives a confident, unsupported conclusion, even if their aggregate scores look similar. The cost of an error depends on the decision that follows it.
Attempts and confidence intervals
OpenAI reports five attempts per task across the 37 settings. That produces 37 × 100 × 5 = 18,500 results, but there are still 100 distinct benchmark tasks, not 18,500 independent tasks. Repeating an item helps show variation in responses; it does not turn that item into a new experiment.
The 95% intervals in the table come from bootstrapping whole tasks, with attempts on the same task grouped together, according to OpenAI. In plain language, this procedure repeatedly resamples tasks to estimate how the reported mean might vary with the task sample, while respecting the fact that repeated attempts on one task share the same underlying question. The intervals do not cover every uncertainty in the benchmark, including whether its tasks represent a particular lab’s work or whether its rubrics capture the consequences of each mistake.
What the examples show
OpenAI’s examples illustrate why evidence interpretation deserves close attention. The report discusses cases where models identify useful features in experimental evidence, alongside failures in which a model reverses a colony-selection conclusion, gives an interpretation that conflicts with microscopy evidence, or overcounts colonies by roughly 12%. These are examples selected in the source, not estimates of how often each error occurs in ordinary use. Source: OpenAI
Three improvements OpenAI illustrates
In one microscopy example, Astra recognized that overlapping detection channels made a cell-death conclusion uncertain. OpenAI contrasts that with GPT-5.6 Sol treating brightness as sufficient evidence. The useful behavior was recognizing what the supplied observation could not establish.
In a cloning example, Astra noticed a plate label identifying a different antibiotic from the one used in the experiment. A plate can look like a valid control while its label changes what it can establish. In another example, Astra recognized a microscope depth scan, or z-stack, instead of explaining changing cell visibility as a biological event. These are OpenAI’s selected examples, not independently repeated Kingy tests.
Correct observations can still produce wrong decisions
OpenAI also shows Astra distinguishing band patterns but assigning the experimental identities incorrectly, selecting the wrong colonies. In another microscopy case, Astra noticed variation with focus yet retained a generalization contradicted by lighter regions in the same field. Its colony-counting example included missed colonies, merged neighbors and marks on empty areas; the reported net error was an overcount of approximately 12% on that plate. The lesson for evaluation is to check whether the final decision actually follows from the observations, controls and uncertainty the model has already identified.
The same answer can contain a plausible explanation and a material mistake. A visually convincing interpretation is particularly difficult to assess if the reader cannot trace the conclusion to the underlying image, measurement or control. The report itself cautions against treating the scores as proof of reliable laboratory judgment.
The results also do not establish why one model scored higher. Differences in model identity, training, tools and inference configuration could matter. A causal claim about any particular architecture or training change would require matched tests designed to isolate it.
A proposed review workflow for lab teams
The following is Kingy’s proposed workflow, not a procedure validated by BioBasicsBench or tested by Kingy. It is intended for qualified scientists assessing an AI assistant against work they already understand.
- Choose a bounded interpretation task. Select representative, appropriately authorized files and define the question, relevant controls and consequences of a wrong answer before asking the model. Protect sensitive data under your institution’s rules.
- Request an auditable answer. Ask the assistant to separate observations from interpretations, point to the evidence behind each conclusion, identify uncertainty and say what information is missing. Preserve the original files and the full response.
- Review against the source material. A qualified scientist checks the response against the original evidence and controls, including any conclusion the model presents confidently. Record unsupported claims, missed ambiguity and requests for clarification.
- Score outcomes by consequence. Track task-level errors and their severity alongside any aggregate score. Separate harmless formatting issues from mistakes that could change a consequential decision. Agree in advance on when a response must be rejected or escalated.
- Retest before relying on a change. Record the model version, settings, tools and inputs. Reassess when those change or when the work moves to a different instrument, sample type or context. Local performance must be established for the intended use.
This process can help a team decide where an assistant is useful for interpretation and where review must be stricter. It does not establish that a model can operate equipment, perform an experiment or replace a scientist.
Safety, biosecurity and the limits of the result
More capable scientific interpretation can be useful, and it also warrants assessment of how a system is used. BioBasicsBench’s reported scores alone do not quantify biological risk, demonstrate dangerous capability or establish that a model is safe for every deployment. Those claims require separate evidence and context. Organizations evaluating use should consider access to sensitive materials, data handling, appropriate oversight and the consequences of errors alongside capability.
The announcement also does not provide a comparable measured human baseline in the inspected material. The numbers therefore do not support a claim that the models match or exceed scientists, or that they are ready for clinical use. Nor does an xhigh research result by itself establish equivalent behavior in a particular public product.
What would make the findings easier to reproduce?
At the time of our November 6 source check, we found no article-body link to a paper, dataset, full rubrics, evaluation code or raw results in the OpenAI announcement. We also inspected the Ligo research index; it did not list a BioBasicsBench companion post then. This was a bounded check, not proof that materials do not exist elsewhere or cannot be released later.
Independent evaluation would be easier to assess with access to task inputs, complete rubrics and per-attempt results, subject to appropriate access and safety decisions. Researchers would also need exact model snapshots, prompts and settings; available tools and file-processing methods; retry and failure rules; and an account of who applied the rubrics and how disagreements were handled. Task selection, domain weighting, contamination controls and an appropriately comparable human baseline would clarify what the overall mean represents.
A follow-up should also report errors by domain and severity, including when a model abstains or expresses uncertainty. A single mean cannot show whether an assistant’s failures are concentrated in the cases that matter most to a particular user.
Frequently asked questions
Does 66.7% mean Astra solved two-thirds of the tasks?
No. It is OpenAI’s reported mean partial-credit reward at xhigh effort. The score can include credit for incomplete answers and does not specify the share of tasks fully solved.
Did Kingy run the benchmark?
No. Not tested by Kingy. This article reports and explains OpenAI’s published results and performs arithmetic on its displayed figures.
Can I use the token table to calculate cost or speed?
Not by itself. The figures cover output tokens per task, not the complete request, input processing, tool use, elapsed time or price. A cost comparison needs the applicable endpoints, configuration and current pricing.
Has an independent group replicated the scores?
Kingy did not find a replication in the bounded sources inspected for this article. Expert review of benchmark items is a different step from an independent rerun of the reported model evaluation.
Sources
- OpenAI: Measuring how AI agents handle everyday wet-lab science — benchmark description, methodology and vendor-reported results, November 6, 2026.
- Ligo — company information.
- Ligo research index — checked for a companion post on November 6, 2026.
- OpenAI announcement on X — discovery pointer.
Revision log
- November 6, 2026: Initial article based on OpenAI’s benchmark report and a bounded check of Ligo’s research index. No independent benchmark run by Kingy.