OpenAI's BioBasicsBench tests wet-lab AI agents

OpenAI introduced BioBasicsBench, a 100-task benchmark developed with Ligo Biosciences to test how AI agents interpret raw evidence from routine wet-lab work. OpenAI says the evaluation measures whether agents can check controls, identify experimental errors and recognize when the evidence is insufficient for a conclusion.

The tasks cover protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. Inputs range from gels and blots to protocols and notes, instrument exports, bench photographs, microscopy, sequencing files, videos, colony plates and reference sheets.

OpenAI says scientists drew the questions from their own experiments and paired each one with an expert-written rubric. Every task was reviewed by another Ligo scientist, an external PhD-level wet-lab scientist and, finally, an internal reviewer.

The benchmark targets a gap between published science and laboratory work in progress. Paper-ready figures are selected for communication, so they may omit failed runs, ambiguous images, conflicting observations and the troubleshooting clues found in raw evidence. An agent working with that material must determine not only what a measurement appears to show, but whether the controls and experimental conditions make the conclusion defensible.

BioBasicsBench focuses on laboratory evidence, while a separate OpenAI LifeVal benchmark covers everyday AI tasks. OpenAI awards criterion-level partial credit for conclusions, measurements, reasoning and recognition of insufficient evidence. Its reported mean reward is the average share of available rubric credit, not the percentage of tasks solved completely.

The company evaluated seven OpenAI models across 37 combinations of model and reasoning effort. With five attempts on every task, the evaluation produced 18,500 graded results. At xhigh reasoning effort, OpenAI reports that GPT-6 Astra received 66.7% mean reward, with a 95% confidence interval of 60.3% to 73.1%. GPT-5.6 Sol received 43.8%, with an interval of 37.6% to 50.0%.

These are provider-run results, not an independent replication. The opened evidence does not include the complete tasks, raw assets, rubrics, scoring implementation or per-attempt outputs needed to reproduce them. The comparison is also limited to OpenAI models and includes no human wet-lab baseline.

OpenAI’s examples show improvement alongside persistent failure. Astra recognized that overlapping fluorescence channels could not support a cell-death conclusion, spotted a negative-control plate using the wrong antibiotic and identified a microscopy sequence as a z-stack rather than a biological disappearance.

OpenAI also found Astra overriding controls in a colony-PCR task, discounting contradictory microscopy observations and overcounting colonies by about 12%. In the colony-PCR example, the model grouped bands visually but reversed their experimental identities. It selected colonies with the wrong plasmid and would have sent the incorrect plasmid into downstream purification. OpenAI cautions that higher benchmark scores do not make current models reliable lab judges.