AMD Instinct MI300X
The Instinct MI300X is AMD's CDNA 3 accelerator, an OAM module with 192 GB of HBM3 at 5.3 TB/s, 304 compute units and a 750 W peak board power, launched 6 December 2023. AMD positions it against the H100 on memory capacity. Eight modules join over Infinity Fabric on a standard OAM baseboard. GPU Smith specs it for inference-heavy deployments where 192 GB per GPU and available capacity matter more than newest-generation datatypes; it has no FP4 or FP6.
- 192 GB per GPU lets a 70B model in FP16 (about 140 GB of weights, by arithmetic) fit on one GPU with room for KV cache, which was the point of the part against 80 GB H100s.
- FP8 is the lowest precision with matrix hardware; there is no FP6 or FP4, so quantised throughput gains stop at 8-bit versus MI350 series.
- FP64 matrix at 163.4 TFLOPS is high for the class, which matters if the same cluster also runs HPC.
- AMD's comparison charts against H100 use AMD's own calculations; benchmark your model rather than relying on ratios.
- At 750 W peak it is the lowest-power OAM part among the AMD Instinct sheets here, which eases chassis cooling.
How much memory does the MI300X have?
192 GB of HBM3 at 5.3 TB/s peak bandwidth with full-chip ECC.
MI300X vs MI325X?
Same CDNA 3 compute (304 CUs, 2.61 PFLOPS FP8); the MI325X raises memory to 256 GB HBM3E at 6 TB/s and board power to 1,000 W.
What is the MI300X power draw?
AMD lists 750 W peak typical board power, delivered over 54V UBB.
Does the MI300X support FP4?
No. FP8 is its lowest-precision floating-point matrix datatype; MXFP4 and MXFP6 arrive with the MI350 series.