Nvidia unveils new type of Ethernet for AI, Grace Hopper ‘superchip’ in full production

Nvidia CEO Jensen Huang made a presentation at the opening of the Computex computer technology conference in Taipei, Taiwan, on Monday, unveiling several new products including a new type of Ethernet switch dedicated to moving high volumes of data for artificial intelligence tasks .

“How do we introduce a new Ethernet that’s backward compatible with everything to turn every data center into a generative AI data center?” Huang as its keynote speaker. “For the first time, we are bringing the capabilities of high-performance computing to the Ethernet market,” Huang said.

The Spectrum-X, known as the Ethernet family, is “the world’s first high-performance Ethernet for AI,” according to Nvidia. A key feature of the technology is that it “does not drop packets,” said Gilad Shiner, senior vice president of networking, at a media briefing.

The first iteration of the Spectrum-X is the Spectrum-4, Nvidia said, which it called “the world’s first 51Tb/sec Ethernet switch built specifically for AI networks”. The switches work in conjunction with Nvidia’s BlueField data processing unit, or DPU, chips that handle data fetching and queuing, and Nvidia fiber-optic transceivers. The company said the switch can route 128 ports of 400-Gigabit Ethernet or 64 800-Gig ports.

Huang placed the Silver Spectrum-4 Ethernet switch chip on stage, noting that it is “huge”, containing a hundred billion transistors on a 90-millimetre x 90-millimetre die manufactured with Taiwan Semiconductor Manufacturing’s “4N” process technology Are. This part runs at 500 watts, Huang said.

The Spectrum-X family is designed to bridge the dichotomy of data centers in two forms. One is what he called “AI factories,” facilities that cost hundreds of millions of dollars for the most powerful GPUs based on Nvidia’s NVLink and Infiniband used for AI training.

The other data center facility is the AI ​​Cloud (multi-tenant, based on Ethernet), which is multi-tenant and focused on serving predictions to AI consumers, which will be served by Spectrum-X.

Spectrum-X is able to “spread traffic throughout the network in the best possible way,” using “a new mechanism for congestion control,” said Shiner, VP Shiner, which prevents the stacking of packets that can occur inside It is possible Memory buffer of a network router.

“We use advanced telemetry to understand latency in the network to identify hotspots before they can cause anything, to keep it congestion-free.”

Nvidia said in prepared remarks that “the world’s top hyperscalers are adopting Nvidia Spectrum-X, including industry-leading cloud innovators.”

Nvidia is building a test-bed computer, it said, at its Israel offices, called Israel-1, a “generative AI supercomputer,” a Dell PowerEdge XE9680 made of H100 GPUs running data across Spectrum-4 switches. using server.

All the news on Computex Available in nvidia’s newsroom,

In addition to the new Ethernet technology, Huang’s keynote showed a new model in the company’s “DGX” series of computers for AI, the DGX GH200, which the company describes as “a new class of large-memory AI supercomputers for large generative AI models”. Presents as. ,

Generative AI refers to programs that produce more than scores, sometimes text, sometimes images, other artifacts, for example, OpenAI’s ChatGPT bot.

The GH200 is the first system with what the company calls its “superchip”, the Grace Hopper board, which houses the Hopper GPU and Grace CPU on a single circuit board, a CPU based on the ARM instruction set, meant to be compatible with Intel and Advanced Micro Devices’ x86 CPUs. compete with.

The first version of the Grace Hopper, the GH200, is “in full production,” Huang said. Nvidia said in a press release that “Global hyperscalers and supercomputing centers in Europe and the US are among the many customers who will have access to GH200-powered systems.”

The DGX GH200 combines 256 of the superchips, Nvidia said, to achieve a combined 1 exaflops — ten to the power of 18, or a billion, billion floating point operations per second — using 144 terabytes of shared memory. According to Nvidia, the computer is 500 times faster than the original DGX A100 machine released in 2020.

The keynote also unveiled MGX, a reference architecture for system makers to quickly and cost-effectively build 100+ server variations. The first partners to use the spec are ASRock Rack, ASUS, GIGABYTE, Pegatron, QCT and Supermicro, with QCT and Supermicro being the first to market with systems in August, Nvidia said.

The entire keynote can be viewed as one replay from nvidia web site,