cpu offload
2 articles
Prefix Caching and KV Cache Offload to CPU RAM and NVMe
When fetching KV cache from CPU RAM or NVMe beats recomputing prefill: Qwen2.5-72B arithmetic, hit-rate economics, vLLM, SGLang, LMCache, Dynamo KVBM.
October 28, 2026• 16 min read
cpu offloadvllm automatic prefix cachingkv cache offload cpu nvme
LLM Training Memory Calculator for ZeRO and FSDP
A 2026 planning model for per-rank LLM training memory, ZeRO and FSDP sharding, GPU count, CPU or NVMe offload, topology rounding, and measured validation.
September 18, 2026• 21 min read
llm training memory calculatorzero memory calculatorfsdp memory calculator