Focus on AI and HPC: NVIDIA GH200 Grace Hopper superchip shows potential in initial tests

The NVIDIA Grace CPU is an alternative to conventional x86 CPUs in the server sector. It is based on the Arm architecture and has up to 144 Neoverse V2 cores. There are two variants of the Grace CPU: firstly, the Grace Hopper Superchip with the H200 GPU and a single Grace chip with 72 cores and HBM memory, and secondly, the Grace Superchip, which contains two Grace CPUs, each with 72 cores – a total of 144 cores – and LPDDR5x memory. With the development of this chip, NVIDIA is aiming to reduce its dependence on the x86 CPU market, in which companies such as AMD and Intel dominate, and to offer its customers its own solution.

Some of the most outstanding features of Grace are:

• High-performance CPU for HPC and cloud computing
• Super-chip design with up to 144 Arm v9 CPU cores
• World’s first LPDDR5x with ECC memory, total bandwidth of 1 TB/s
• SPECrate2017_int_base over 740 (estimated)
• Coherent interface with 900 GB/s, 7 times faster than PCIe Gen 5
• 2 times higher packaging density than DIMM-based solutions
• 2x higher performance per watt than today’s leading CPUs
• Supports all NVIDIA software stacks and platforms, including RTX, HPC, AI and Omniverse

NVIDIA has claimed in official benchmarks that the Grace CPU would offer up to double the performance of Intel Sapphire Rapids and AMD Genoa CPUs at the same power. It has also been shown to be up to 3.5 times more efficient than the latest AMD EPYC Milan CPUs. Phoronix has now conducted extensive Linux-based HPC benchmarks to verify these claims.

The NVIDIA Grace CPU appears to be almost on par with Intel’s top of the line 60-core Emerald Rapids Xeon CPU and AMD’s 64-core EPYC Genoa chips. This is to be expected as the Grace CPU has a higher core count of 72 cores. However, as this is a new chip, it is expected that performance will be further improved as NVIDIA’s Arm architecture is specifically optimized for HPC and server applications.
The most important test, namely the chip’s power consumption and efficiency, was not performed as the tech outlet reports that power monitoring support is not yet available for the GH200 Grace Hopper Superchip.

However, this support will be added as soon as it is available. The Grace Superchip with two Grace CPUs and 144 cores is rated at up to 500W total power, so one could estimate a power consumption of 200-250W per CPU, which is significantly lower than the 320-350W TDP ratings of top-of-the-line AMD EPYC and Intel Xeon CPUs.

All in all, NVIDIA’s Grace CPU seems to be a promising entry for Team Green into the Arm CPU segment. The 144-core Grace superchip will be particularly exciting as we see the performance boost from double the number of cores and the scaling of performance.

Source: Phoronix