Intel Gaudi 3 AI Accelerator
Gaudi 3 is Intel's dedicated AI accelerator: two 5nm compute dies with 8 matrix engines (MME), 64 programmable tensor processor cores (TPC), 128 GB of HBM2e at 3.7 TB/s, and 24 x 200 GbE RoCE v2 ports on the chip itself, so scale-up and scale-out both run over standard Ethernet instead of a proprietary fabric. It ships in two form factors: the HL-338, a 600 W, dual-slot, passively cooled PCIe Gen5 card for standard servers, and the HL-325L, a 900 W OCP OAM 2.0 module that goes eight to a node on the HLB-325 universal baseboard. GPU Smith treats the HL-338 as the part to evaluate for single-node inference in existing server fleets, and the OAM/HLB-325 path as the Ethernet-fabric alternative to an 8-GPU HGX node. The catch is software: it runs PyTorch, vLLM and Hugging Face through Intel's Gaudi stack, not CUDA.
- The headline FP8/BF16 number changed between revisions of Intel's own technical paper: 1,835 TFLOPS in the 2024 copy, 1,678 TFLOPS in the March 2025 revision, for both OAM and PCIe. Use 1,678 and treat both as peak, not sustained. Intel's table gives the 600 W PCIe card the same peak as the 900 W OAM; sustained clocks at 600 W are not published, so benchmark before assuming parity.
- Ethernet everywhere is the real architectural difference. The 24 on-die 200 GbE ports carry both scale-up and scale-out, so an 8-OAM node plugs into standard 800 GbE switches with no InfiniBand or NVLink switch. On the PCIe card that only applies within a quad via the top board; beyond four cards, traffic goes through a host NIC.
- An early 2024 HL-338 preview reported external QSFP-DD ports on the card; Intel's current product brief and technical paper list scale-out through the host NIC only. Spec the current part from the brief, not from launch photos.
- There is no FP4 support, and 128 GB of HBM2e at 3.7 TB/s is a memory generation behind the HBM3 and HBM3e on H100 SXM and later NVIDIA parts. The case for Gaudi 3 is price per accelerator and an open Ethernet fabric, not peak throughput against Blackwell-class parts.
- Roadmap risk is real. Intel still sells Gaudi 3, but Falcon Shores became an internal development vehicle, Crescent Island (inference GPU, 160 GB LPDDR5X) targets customer sampling in H2 2026, and Jaguar Shores is the longer-term GPU architecture. Intel's named next parts are GPUs, not Gaudi, so do not expect code built on Gaudi-specific kernels to carry forward.
How much does Intel Gaudi 3 cost?
Intel listed a kit of eight Gaudi 3 OAM accelerators on a universal baseboard at $125,000 in June 2024, about $15,600 per accelerator. Pricing for the HL-338 PCIe card is not published; resellers quote it on request.
What is the difference between the HL-338 and the HL-325L?
Both use the same Gaudi 3 chip with 128 GB HBM2e. The HL-338 is a 600 W, dual-slot, passively cooled PCIe 5.0 card for standard servers, scaling to four cards over a top board and beyond that through a host NIC. The HL-325L is a 900 W OCP OAM module mounted eight to an HLB-325 baseboard, with all 24 on-die 200 GbE ports wired for all-to-all scale-up and scale-out.
Does Gaudi 3 use NVLink or InfiniBand?
Neither. Each chip has 24 x 200 GbE RoCE v2 ports built in, and Intel's cluster reference design uses standard 800 GbE leaf and spine switches. On the eight-OAM baseboard, 21 ports per chip go to the other seven chips and 3 go to six OSFP 800 GbE scale-out connectors.
Does Gaudi 3 run CUDA?
No. It runs PyTorch, vLLM and Hugging Face models through Intel's Gaudi software stack. Standard model code usually ports, but custom CUDA kernels must be rewritten, and Intel's next accelerators (Crescent Island, Jaguar Shores) are a different architecture.