NVIDIA H100 SXM
The H100 SXM is NVIDIA's Hopper-generation compute GPU in the socketed SXM5 form factor, sold only as a component of an HGX baseboard (4 or 8 GPU) inside partner or DGX systems, never as a standalone card. It is the workhorse part behind most of the Hopper-era training and inference fleet still in production: 80 GB of HBM3 at 3.35 TB/s, a fourth-generation Transformer Engine, and 900 GB/s of NVLink to the other GPUs on the baseboard. As of mid-2026 it is a prior-generation part: NVIDIA's shipping frontier is Blackwell Ultra (B300) and Rubin has entered production, so H100 buys now are almost always driven by compatibility with an existing Hopper fleet, software stack, or by price rather than by peak performance. It remains a rational choice when a workload fits in 80 GB and the buyer can get it cheaper per FLOP than a Blackwell or H200 seat.
- 80 GB per GPU is the real ceiling on this part: a 70B-parameter model at FP16 with KV cache headroom fits, but large MoE or long-context serving workloads will push you toward H200 or B200 for memory capacity, not raw compute.
- Sparsity numbers on the datasheet (3,958 TFLOPS FP8) assume structured 2:4 sparsity that most production models do not hit; budget compute against the dense figures (1,979 TFLOPS FP8) for realistic throughput planning.
- At 700 W per GPU and 8 GPUs per node, air cooling is workable but liquid cooling meaningfully lowers fan power and lets you run denser racks; factor this into total cost of ownership, not just the GPU price.
- If the workload needs more than 80 GB per GPU or you are buying new today, price out H200 SXM before committing to H100: the memory bandwidth and capacity jump often beats the price delta once you count fewer GPUs needed per model shard.
- Because H100 SXM is not sold standalone, the actual constraint on lead time is usually the HGX baseboard and chassis, not the silicon itself; ask vendors for baseboard and NVSwitch stock separately from GPU die allocation.
Is the H100 still worth buying in 2026?
Only if it is cheaper per FLOP than H200 or a Blackwell part for your workload, or you need it to match an existing Hopper fleet. NVIDIA's current shipping frontier is Blackwell Ultra with Rubin now in production, so H100 is a value or compatibility buy, not a performance one.
How much does an H100 SXM cost?
Reported street prices run roughly $24,000 to $40,000 per GPU depending on channel and volume, with an 8-GPU HGX server in the $216,000 to $300,000 range. Used SXM5 cards on the secondary market have been reported as low as $6,000 to $15,000.
Can I buy a single H100 SXM card?
Not really. SXM5 is a socketed module that ships as part of an HGX baseboard or a complete DGX/OEM server; if you need a single discrete card, look at the PCIe H100 instead.
H100 vs H200: which should I pick?
Pick H100 when budget or fleet compatibility with existing Hopper systems dominates and 80 GB is enough for the model. Pick H200 when you need more memory capacity or bandwidth per GPU, since compute throughput on the two parts is otherwise identical.