OpenAI introduces MedChemBench to test AI on medicinal chemistry
MedChemBench draws 99 problems from papers and patents. Its overall score averages the share of available grading points earned, with each problem weighted equally.
OpenAI introduced MedChemBench on November 20 to measure how AI models interpret molecular structures and experimental results in medicinal chemistry. In an accompanying preprint dated November 9, OpenAI researcher Jeremy Li and his co-authors describe the benchmark, which covers 99 problems drawn from 117 papers and patents, with 761 distinct molecular structures assessed.
The problems revisit decisions from past research. Models typically receive a research goal, molecular structures and selected experimental results. The authors withhold later findings from the questions and use those findings to help grade the answers. The announcement and preprint include examples but provide no download for the full set of problems or the grading code.
The report scores each response by the share of available points it earns, with partial credit on some criteria. For each model and reasoning setting, the authors average ten retained responses per problem, then give each problem equal weight. OpenAI says each answer was generated in a single turn without tools, feedback or new experiments. The authors say the models may have encountered the source material before.
At the xhigh reasoning setting, OpenAI reported 84.5% for GPT-6.2 Astra and 83.3% for GPT-6 Astra in its announcement. OpenAI did not cite an independent replication of either score in the announcement. The preprint covers 11 models and excludes GPT-6.2 Astra.
For GPT-6 Astra at that setting, the report gives 95.37% of available points for reconstructing structures, 91.05% for explaining findings, 50.42% for ranking and selecting compounds, and 47.02% for numerical predictions. Each component score uses the problems that assess that skill.
More news

Artificial Analysis puts Gemini 4 Neon one point above Carbon at lower standard-price cost

Artificial Analysis tightens endpoint test as 24 of 103 trail references

Artificial Analysis and Google DeepMind launch benchmark for visual knowledge work
