Intel Unveils 'Gaudi 3' AI Chip

Intel has introduced its latest artificial intelligence (AI) chip, the ‘Gaudi 3,’ during a live-streamed keynote at the company’s Intel Vision 2024 conference in Phoenix, Arizona. The chip is targeted at enterprises and aims to automate complex enterprise tasks.

A Rival to Nvidia’s H100

According to Intel CEO Pat Gelsinger, the Gaudi 3 chip is almost twice as fast as Nvidia’s H100 GPU when it comes to training AI models, such as the TensorRT large language model. Additionally, the Gaudi 3 reportedly performs inference tasks 50% faster than the H100. Inference is a process in which a trained neural net makes predictions in response to real questions.

Success in Benchmark Tests

The Gaudi chip family has demonstrated its capabilities in recent benchmark tests against Nvidia. The Gaudi 2 chip, in particular, was the only data center chip that competed with the H100 in the latest MLPerf competition held by the MLCommons. This competition involved making predictions using Meta’s open-source Llama 2 70-billion large language model.

Gaudi 3: An Overview

The Gaudi 3 is the third generation of Intel’s dedicated chip for performing AI training and inference. It consists of 64 separate tensor cores on die, which accelerate matrix multiplications at the core of AI processing. These cores are aided by eight discrete “matrix math engines.” The chip also uses 96 megabytes of fast on-chip SRAM cache memory, and a further 128 gigabytes of external “HBM3e” memory.

Potential for Enterprise Applications

During the event, Gelsinger, along with industry leaders like Michael Dell, emphasized the applicability of the Gaudi 3 for enterprise tasks. He outlined three stages of AI development: the current “Age of AI Co-pilots,” the upcoming “Age of AI agents,” and the future “Age of AI Functions.” Gelsinger believes that the Gaudi 3 will aid the progression through these stages, ultimately leading to the automation of complex, enterprise-wide outcomes.

A Trillion Operations per Second

The Gaudi 3 is capable of achieving 1.84 teraFLOPs, or a trillion floating-point operations per second, when working on 8-bit floating point math operations. This performance metric is widely used to measure chip performance. According to Gelsinger, this performance, coupled with automation capabilities, could potentially enable the creation of the first one-person, billion-dollar company.