MLCommons adds agentic and recommendation tests to MLPerf Training v6.1
MLCommons published 250 MLPerf Training v6.1 results and added agentic post-training and generative recommendation workloads as submitted systems scaled to 10,240 GPUs and 8,192 TPUs.
MLCommons released MLPerf Training v6.1 results with 250 test results from 153 submitted systems. The round adds standardized tests for agentic large-language-model post-training and generative recommendation, while its largest submissions show how quickly very large accelerator clusters can reach fixed training targets.
In the benchmark’s Closed division, which holds workloads and quality targets constant, NVIDIA’s system using 10,240 GB300 GPUs reached the DeepSeek-V3 target in 1.38 minutes. Microsoft’s Azure submission using 10,240 GB200 GPUs recorded 2.36 minutes, and Google’s system using 8,192 TPU7x Ironwood chips recorded 1.70 minutes. Those figures are workload-specific time-to-train scores, not measures of general-purpose performance, system cost or price-performance. A separate 10,000-plus-GPU GB300 infrastructure deployment provides context for the physical scale of the largest submission.
The new Qwen3.5-397B workload measures the time required to improve a 397-billion-parameter mixture-of-experts model on software-repair tasks. It uses reinforcement learning with verifiable rewards and Group Relative Policy Optimization, or GRPO, and stops when the model reaches a fixed pass@4 target of 0.69. Seven submitters entered 29 results for the workload.
The other addition, DLRMv4-HSTU, replaces DLRM-DCNv2 as the suite’s recommendation benchmark. It uses a sequence model called HSTU and the Yambda-5B dataset, which contains about 4.79 billion interactions. Its embedding tables occupy approximately 560 GiB after cross-feature tables are included. Eight submitters entered 20 results for the workload.
The round also shows how time to the fixed DeepSeek-V3 target changed as one NVIDIA GB300 submission scaled. Official results recorded 31.40 minutes with 256 GPUs, 8.03 minutes with 1,024 GPUs and 1.38 minutes with 10,240 GPUs. MLCommons calculated reductions of 3.9 times and 5.8 times between those configurations. Those ratios are the organization’s derived analysis, not fields in the official result records.
MLCommons counted 117 of the round’s 153 systems, or 76%, as multi-node, compared with 72 of 118 systems, or 61%, in v6.0. It also said the median accelerator count per Closed result increased from eight in earlier rounds to 32 in v6.1. Twelve systems used at least 1,000 accelerators, while 57 systems, or 37%, were cloud-based.
Submitters attributed some gains to changes including ROCm optimization, AI-generated kernels, activation storage and drivers. The published benchmark scores verify the corresponding times to the workload targets, but they do not independently isolate those changes as the cause of the gains.
The results contain no normalized acquisition-price, cloud-job-cost or total-cost-of-ownership data. Energy comparisons are also limited: MLCommons said TTA supplied the round’s only energy measurement, leaving no broad cross-system energy-efficiency comparison for v6.1.
More news

Neysa deploys AMD MI350X cluster in India

NVIDIA reports $115.8 billion in fiscal 2027 third-quarter revenue

Lambda deploys NVIDIA Photonics switches across 10,000-plus GPUs
