AI Speed Benchmark Adds Tests for Search and Coding Agents
MLCommons, the open engineering consortium behind the MLPerf benchmarks, published results for its MLPerf Inference v6.1 suite on Sept. 16, 2026. Thirty organizations submitted, which the consortium said is a record for the suite, and two tests were new: one for retrieval-based question answering, one for coding agents on a single edge device.
In its announcement, MLCommons said the suite measures system performance in an architecture-neutral and reproducible way, so that customers buying and deploying AI systems can compare them on published data.
The first new test, End-to-End Retrieval-Augmented Generation, times a whole question-answering pipeline rather than a single model: a query is converted for search, a retriever pulls candidate passages from a database, a further step narrows the list, and one or more language models write the answer. The second, Edge Agentic Inference, runs a multi-step coding job on one device serving one user, where each query depends on the ones before it and memory, power and context are fixed.
Entries are run by the submitters themselves under MLCommons' rules and reviewed by the consortium before publication. Six were first-time submitters, and the round carried the largest system ever entered in MLPerf Inference, at 512 accelerators, the chips that do the AI computing. One entry combined accelerators from two different vendors; another was spread across the Pacific Ocean.
On the DeepSeek R1 workload, MLCommons reported, the best per-accelerator result in the suite's server scenario was 5.7 times the best figure from v5.1 a year earlier. That is the strongest single submission on one of the suite's tests, not a suite-wide measure, and the consortium publishes each submitter's own account of its entry alongside the tables.
More than half the submitters used a new harness, the software that feeds queries to the system being tested, which sends them over standard APIs from a separate client. MLCommons said that harness is the basis of a coming suite, MLPerf Endpoints, which will replace Inference for datacenter benchmarking.
