NVIDIA has released its first data center CPU named NVIDIA Grace, an Arm-based processor. According to NVIDIA, it will deliver 10 times the performance of today’s fastest servers on the most complex artificial intelligence (AI) and high-performance computing (HPC) workloads.
The result of more than 10,000 engineering years of work, the NVIDIA Grace CPU is designed to address the computing requirements for the world’s most advanced applications including natural language processing, recommender systems and AI supercomputing. The NVIDIA Grace CPU is capable of analyzing enormous datasets that require both ultra-fast compute performance and massive memory.
The NVIDIA Grace CPU combines energy-efficient Arm CPU cores with an innovative low-power memory subsystem to deliver high performance with great efficiency. Availability is expected in the beginning of 2023.
Highly Specialized Processor
“Leading-edge AI and data science are pushing today’s computer architecture beyond its limits – processing unthinkable amounts of data,” said Jensen Huang, founder and CEO of NVIDIA. “Using licensed Arm IP, NVIDIA has designed Grace as a CPU specifically for giant-scale AI and HPC. Coupled with the GPU and DPU, Grace gives us the third foundational technology for computing, and the ability to re-architect the data center to advance AI. NVIDIA is now a three-chip company.”
While the vast majority of data centers are expected to be served by existing CPUs, Grace – named for Grace Hopper, the U.S. computer-programming pioneer – will serve a niche segment of computing.
The NVIDIA Grace CPU is a highly specialized processor targeting workloads such as training next-generation NLP models that have more than 1 trillion parameters. When tightly coupled with NVIDIA GPUs, a Grace CPU-based system would deliver 10 times faster performance than today’s NVIDIA DGX-based systems, which run on x86 CPUs.
Arm Data Center Technology
NVIDIA is introducing their Grace CPU as the volume of data and size of AI models are growing exponentially. Today’s largest AI models include billions of parameters and are doubling every two-and-a-half months. Training them would require a new CPU that can be tightly coupled with a GPU to eliminate system bottlenecks. NVIDIA built the Grace CPU by leveraging the flexibility of Arm’s data center architecture.
“As the world’s most widely licensed processor architecture, Arm drives innovation in incredible new ways every day,” said Simon Segars, CEO of Arm. “NVIDIA’s introduction of the Grace data center CPU illustrates clearly how Arm’s licensing model enables an important invention, one that will further support the incredible work of AI researchers and scientists everywhere.”
Underlying Grace’s performance is fourth-generation NVIDIA NVLink interconnect technology, which provides a record 900 GB/s connection between Grace and NVIDIA GPUs to enable 30x higher aggregate bandwidth compared to today’s leading servers.
NVIDIA Grace will also utilize an “innovative” LPDDR5x memory subsystem that would deliver twice the bandwidth and 10 times better energy efficiency compared with DDR4 memory. In addition, the new architecture provides unified cache coherence with a single memory address space, combining system and HBM GPU memory to simplify programmability.
NVIDIA Grace will be supported by the NVIDIA HPC software development kit and the full suite of CUDA and CUDA-X libraries, which accelerate more than 2,000 GPU applications, speeding discoveries for scientists and researchers working on the world’s most important challenges.