LLM Evaluation & Leaderboards
News on LLM evaluation and benchmarks, covering new tests, methodology changes, leaderboard results, and whether findings can be reproduced.
Explore how researchers and engineering teams test language models before and after deployment. Coverage looks at benchmark design, meaningful comparisons, leaderboard changes, and which results teams can reproduce.
Latest coverage
Articles
NewsVals finds prompt framing shifts AI targeting in simulations
NewsPerplexity releases BrowseSafe++ benchmark for browser-agent prompt injection
NewsCWE-bench v2 opens 100 public security-repair tasks
NewsAllen Institute for AI releases StateBench for long-document state tracking
NewsAi2 releases XYBench to test how assistants handle mistaken premises
NewsApple study finds MMLU weakly tracks LLM judge quality
NewsMBZUAI introduces MedRoundsQA benchmark for diagnostic conversations
NewsVals finds DeepSeek trails GPT-6 Sol Daybreak by 8 points in simulated cyber test
NewsOpenAI introduces LifeVal benchmark for everyday AI tasks
NewsProperty Finder rebuilds AI benchmark, reports 86% lower inference cost
NewsIBM and Kaggle launch maintained ScarfBench Java migration leaderboard
NewsiMerit details Scholars workflows for specialized AI model evaluation
Videos
1:02:51Tech TalksOperationalizing LLMs: From Prototype to Production
Yogiraj Awati·Sep 17, 2026
48:04Tech TalksBuilding Trustworthy Financial AI: Governance, Bias, and Mechanistic Insights
Fabrizio Dimino·Sep 17, 2026
4:40AI Demo Jam: Future AGI
Dmytro Spodarets·Apr 15, 2025
1:13:21Evaluating LLM Models for Production Systems: Methods and Practices
Dmytro Spodarets·Feb 28, 2024
1:01:39GPT on a Leash: Evaluating LLM-based Apps & Mitigating Their Risks
Dmytro Spodarets·Dec 15, 2023
42:58Large Language Model Evaluations - What and Why
Dmytro Spodarets·Jul 24, 2023