MLCommons, an artificial intelligence benchmarking group, has unveiled a new set of tests and outcomes evaluating the efficiency of cutting-edge hardware in running AI applications and promptly responding to user queries.
The benchmarks, released on Wednesday, shed light on the rapidity with which AI chips and systems can process data and generate responses from robust AI models.
The latest benchmarks introduced by MLCommons focus on gauging the swiftness of AI chips and systems in producing responses from data-packed AI models. These benchmarks serve as indicators of how promptly AI applications, such as ChatGPT, can provide answers to user inquiries.
One of the new benchmarks, titled Llama 2, is specifically designed to measure the speed of question-and-answer scenarios for large language models. Developed by Meta Platforms, Llama 2 boasts an impressive 70 billion parameters, showcasing the advancements in AI technology.
Additionally, MLCommons has expanded its suite of benchmarking tools with the inclusion of a second text-to-image generator, named MLPerf. This generator, based on Stability AI’s Stable Diffusion XL model, further enriches the evaluation process for AI performance.
Servers equipped with Nvidia’s H100 chips, utilized by tech giants such as Alphabet’s Google, Supermicro, and Nvidia itself, emerged as frontrunners in both new benchmarks in terms of raw performance. Notably, various server manufacturers presented designs featuring Nvidia’s L40S chip, albeit less potent than the H100.
Krai, a server builder, submitted a design for the image generation benchmark employing a Qualcomm AI chip, which boasts significant power efficiency compared to Nvidia’s latest processors.
Intel also entered the arena with its Gaudi2 accelerator chips, presenting results deemed as “solid” by the company. While raw performance is crucial, the energy consumption of advanced AI chips remains a significant concern. Achieving an optimal balance between performance and energy efficiency poses a notable challenge for AI companies.
In the dynamic landscape of AI technology, benchmarks provided by organizations like MLCommons serve as pivotal tools for assessing and advancing the capabilities of AI hardware, ensuring strides towards more efficient and sustainable AI applications.