fsdp memory calculator

1 article

LLM Training Memory Calculator for ZeRO and FSDP

A 2026 planning model for per-rank LLM training memory, ZeRO and FSDP sharding, GPU count, CPU or NVMe offload, topology rounding, and measured validation.

September 18, 2026• 21 min read
llm training memory calculatorzero memory calculatorfsdp memory calculator