AMD Instinct MI350P
The Instinct MI350P is AMD's CDNA 4 accelerator in a dual-slot, full-height, passively cooled PCIe card, announced in May 2026. It carries 144 GB of HBM3E at 4 TB/s and 128 compute units, in a 600 W envelope that can be configured down to 450 W, so it fits standard air-cooled 19-inch servers instead of requiring an OAM baseboard. Cards talk to each other over PCIe 5.0 only; AMD configurations run from one to eight cards per node, reported. It is the AMD answer to the PCIe H200 NVL and RTX PRO 6000 Server Edition class of part, and GPU Smith specs it for single-node private inference where HBM capacity and ROCm support matter more than a scale-up fabric.
- No high-speed card-to-card link: tensor-parallel splits across cards ride PCIe 5.0 through the host. Run one model per card, or pipeline-parallel across cards, rather than wide tensor parallelism in one node.
- 144 GB on a single card is the point. Many 70B-class models in FP8, and some larger ones in MXFP4, fit without any parallelism, which simplifies serving.
- Peak figures are theoretical matrix throughput at the engine clock. AMD's own page does not publish delivered numbers; compare with measured results on your model, not the FP4 headline.
- ROCm is the software decision, not the silicon. Check that your serving stack (vLLM, SGLang, your kernels) is supported on CDNA 4 before committing.
- The 450 W configuration trades some peak for rack power headroom; confirm the throughput hit on your workload before standardising on it.
How much memory does the MI350P have?
144 GB of HBM3E on a 4096-bit interface with 4 TB/s peak bandwidth, with full-chip ECC.
How much does the MI350P cost?
AMD has not published a price. Unconfirmed third-party estimates run $30,000 to $40,000 per card. Get an OEM quote.
MI350P vs MI355X?
The MI355X is the OAM module: 288 GB HBM3E, 8 TB/s, 256 compute units and 1,400 W, built for 8-GPU baseboards with Infinity Fabric. The MI350P is a half-scale PCIe card (144 GB, 4 TB/s, 128 CUs, 600 W) for ordinary servers with no scale-up fabric.
Does it need special server hardware?
Yes. It is passively cooled and draws up to 600 W through a 12V-2x6 connector, so it belongs in an OEM-qualified chassis with enough airflow and power delivery.