AVCC and MLCommons Release New MLPerf Automotive v0.5 Benchmark Results
New benchmark opens a window into critical performance data of automotive AI systems
SAN FRANCISCO, Aug. 27, 2025 (GLOBE NEWSWIRE) -- Today AVCC® and MLCommons® announced new results for their new MLPerf® Automotive v0.5 benchmark suite, which delivers machine learning (ML) system performance benchmarking in an architecture-neutral, representative, and reproducible manner. MLPerf Automotive is the result of a broad, cross-industry collaboration that provides new and critical performance data for stakeholders who procure AI systems for automobiles.
MLPerf Automotive: A Cross-industry Partnership to Provide Critical Performance Data
The MLPerf Automotive benchmark was designed by a working group composed of technical experts from thirteen organizations representing both the AI community and the automotive manufacturing industry, including Ambarella, ARM, Bosch, C-Tuning Foundation, CeCaS, Cognata, Motional, NVIDIA, Qualcomm, Red Hat, Samsung, Siemens EDA, UC Davis, and ZF Group.
The v0.5 version of the benchmark includes three performance tests:
- 2-D object recognition and segmentation
- 3-D object recognition
“Many of the key scenarios for AI in automotive environments relate to safety, both inside and outside of a car or truck,” said James Goel, Automotive Working Group co-chair. “AI systems can train on 2-D images to be able to detect objects in a car’s blind spot or to implement adaptive cruise control. Having high-quality, 8-megapixel imagery – as we do in this benchmark – ensures that the results reflect real-world demands on these systems. In addition, 3-D imagery is critical for training and testing collision avoidance systems, whether assisting a human driver or as part of a fully automated vehicle.”
The benchmark suite implements two scenarios for measuring performance: a “single stream” of requests that measures raw performance and throughput; and a “constant stream” where requests arrive at intervals, providing important data on system latency.
“As vehicles become increasingly intelligent through AI integration, every millisecond counts when it comes to safety,” said Kasper Mecklenburg, Automotive Working Group co-chair and principal autonomous driving solution engineer, Automotive Business, Arm. “That’s why latency and determinism are paramount for automotive systems, and why public, transparent benchmarks are crucial in providing Tier 1s and OEMs with the guidance they need to ensure AI-defined vehicles are truly up to the task.”
Datasets for these benchmark tests were sourced through partnerships with Cognata and Motional. Cognata provided 8-megapixel imagery for the 2-D object recognition and segmentation tests, and Motional provided its nuScenes dataset for the 3-D object recognition test.
"Reliable autonomy depends on transparent, realistic benchmarks,” said Danny Atsmon, Founder and CEO of Cognata. “By contributing Cognata’s 8-megapixel imagery dataset, we are not only strengthening MLPerf Automotive v0.5 but also reinforcing Cognata's role as a trusted provider of benchmark-quality datasets that accelerate safe, large-scale adoption of ADAS and AV systems.”
“Since its release in 2019, nuScenes has played an important role in many critical advancements by the autonomous vehicle and research communities,” said Sourabh Vora, Senior Director of Engineering at Motional. “The first publicly available dataset of its kind, nuScenes pioneered an industry-wide culture of data-sharing and collaboration, and continues to serve as the industry standard for datasets today. Motional is proud to continue supporting the development of safe autonomous systems with the use of nuScenes for the MLPerf Automotive benchmark suite.”
Valuable performance data, immediately relevant for decision-makers
“The MLPerf Automotive benchmark suite is the product of a unique and essential collaboration, bringing the expertise of the computing, sensor, AI, and automotive communities together towards a common goal: transparent and trustworthy performance data on these emerging automotive AI systems,” said David Kanter, head of MLPerf at MLCommons. “I thank all our partners and collaborators who contributed to this bold first step. I also welcome the submitters in the first round of the benchmark: GateOverflow and Nvidia.”
Markus Tremmel, Bosch ADAS Chief Expert, said, “Selecting the right AI acceleration solution is the basis of every successful ADAS/AD system development. MLPerf® Automotive will greatly enhance and simplify this selection process by establishing realistic automotive benchmarks representing high-resolution system scenarios and modern AI/ML network architectures.”
"The release of MLPerf Automotive v0.5 is a major milestone, showing what’s possible when leaders from AI, semiconductor, and automotive industries work together. With transparent, reproducible benchmarks, OEMs and suppliers can confidently evaluate solutions for next-generation safety-critical automotive systems,” said Stephen Miller, AVCC Technical Chair and Technical Expert, ADAS Product Management at Bosch. “AVCC is proud to partner with MLCommons to support this important joint initiative, which strengthens the foundation for innovation across the global automotive ecosystem.”
View the results
To view the results for MLPerf Automotive, please visit the Automotive benchmark results page.
About AVCC
AVCC is a global automated and autonomous vehicle (AV) consortium that specifies and benchmarks solutions for AV computing, cybersecurity, functional safety, and building block interconnects. AVCC is a not-for-profit membership organization building an ecosystem of OEMs, automotive suppliers, and semiconductor and software suppliers in the automotive industry. The consortium addresses the complexity of the intelligent-vehicle software-defined automotive environment and promotes member-driven dialogue within technical working groups to address non-differentiable common challenges. AVCC is committed to driving the evolution of autonomous and automated solutions up to L5 performance. For additional information on AVCC membership and technical reports, please visit www.avcc.org or email outreach@avcc.org.
About MLCommons
MLCommons is the world’s leader in AI benchmarking. An open engineering consortium supported by over 125 members and affiliates, MLCommons has a proven record of bringing together academia, industry, and civil society to measure and improve AI. The foundation for MLCommons began with the MLPerf benchmarks in 2018, which rapidly scaled as a set of industry metrics to measure machine learning performance and promote transparency of machine learning techniques. Since then, MLCommons has continued using collective engineering to build the benchmarks and metrics required for better AI – ultimately helping to evaluate and improve AI technologies’ accuracy, safety, speed, and efficiency.
For additional information on MLCommons and details on becoming a member, please visit MLCommons.org or email participation@mlcommons.org.
Legal Disclaimer:
EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.
