In a major announcement at the Intel Vision event, Intel unveiled its highly anticipated Intel Gaudi 3 AI accelerator, aimed at challenging Nvidia in the competitive AI training and inference markets. The new accelerator promises significant improvements over its predecessor, Gaudi 2, and competitors, particularly in processing BF16 data types crucial for AI workloads. Manufactured using a 5nm process technology, Gaudi 3 incorporates architectural advancements like more TPCs and MMEs, providing the computing power necessary for parallel processing of AI operations and reducing training and inference times for complex AI models.
With an increased number of Matrix Math Engines and Tensor Cores compared to Gaudi 2, Gaudi 3 boasts an FP8 precision throughput of 1835 TFLOPS – double the performance of its predecessor. It also features 128GB of HBMe2 memory, offering substantial memory bandwidth and onboard static RAM to efficiently process large datasets essential for training and running large AI models. Additionally, the new accelerator prioritizes high-speed, low-latency networking with twenty-four 200Gb Ethernet ports, enhancing system connectivity for efficient scaling of AI compute clusters without proprietary networking technologies.
Intel projects robust performance improvements with Gaudi 3 across key areas relevant to AI training and inference tasks, particularly for LLMs and multimodal models. The company predicts that Gaudi 3 will outperform Nvidia’s competing products in terms of training speed, inference throughput, and power efficiency for various parameterized models, with an average 50% faster training time and superior inference performance. Intel’s strategic move with Gaudi 3 aims to address the increasing demand for advanced AI compute solutions and challenge Nvidia’s dominance in the market.
The Gaudi 3 AI accelerator represents a significant advancement for Intel in the AI accelerator market, positioning the company as a key player against competitors like Nvidia. By offering substantial performance improvements, open community-based software, and standard Ethernet networking, Intel is aligning with industry trends towards open standards and interoperability. Partnerships with leading OEMs like Dell Technologies, HPE, Lenovo, and Supermicro further set Intel up for success in the accelerator market, potentially leading to significant growth if Intel can deliver on its promised performance claims.
Beyond the current generation of AI accelerators, Gaudi 3 also sets the stage for Intel’s future GPU, Falcon Shores, by integrating the Intel Gaudi and Intel Xe IP with a single GPU programming interface to enhance capabilities in AI and HPC. Overall, the launch of the Gaudi 3 AI accelerator underscores Intel’s technological advancements, strategic market positioning, and commitment to addressing the evolving needs of the AI industry. By challenging the status quo and positioning itself as a leader in AI infrastructure, Intel is poised to make a significant impact in the next wave of AI innovation.
