MLCommons, an industry organization specializing in artificial intelligence performance evaluation and machine learning hardware, has added the latest results of artificial AI and ML accelerators to its database, and in essence released the first performance data standard test set of Nvidia's H100 and Biren's BR104 computing GPUs obtained through the industry. The results are compared with those obtained on Intel's Sapphire Rapids, Qualcomm's AI 100 and Sapeon's X220.
MLPerf of MLCommons is a set of training and reasoning benchmarks, recognized by dozens of support organizations and companies that submit their hardware test results to the MLPerf database. MLPerf Conference version 2.1 benchmark test set includes data center and edge use scenarios and image classification (ResNet 50 v1.5), natural language processor (BERT Large), speech recognition (RNN-T), medical imaging (3D U-Net), object detection (RetinaNet) and recommendation (DLRM).
The machines participating in these tests are evaluated in two modes: in the server mode, queries arrive in bursts, and in the offline mode, all data is provided at one time. Therefore, their performance is obviously better in the offline mode. In addition, suppliers can submit results obtained under two conditions: in the closed category, everyone must run mathematically equivalent neural networks, and in the open category, they can modify them to optimize their hardware, IEEE Spectrum report.
The results obtained in MLPerf not only describe the pure performance of the accelerator (for example, one H100, one A100, one Biren BR104, etc.), but also describe their scalability and performance per watt to draw a more detailed picture. All results can be viewed in the database, but Nvidia summarizes the performance results of each accelerator based on the information submitted by itself and the third party.
Nvidia's competitors have not yet submitted all the results, so the charts published by Nvidia are missing some results. However, we can still draw some very interesting findings from the table published by Nvidia (but remember that Nvidia is a stakeholder here, so everything should be reserved).
