gpu memory
3 articles
FP16 vs FP8 vs INT4 Inference Cost: Quality and GPU Fit
A 2026 guide to quality-adjusted GPU inference cost, comparing FP16/BF16, FP8 and exact INT4/FP4 recipes with a compatibility crosswalk, integer GPU-fit examples, acceptance gates and a Python calculator.
October 15, 2026• 24 min read
fp16 vs fp8 vs int4 inference costfp8 inferenceint4 quantization
vLLM KV Cache Capacity Planning: Context and Concurrency
A 2026 engineering guide to vLLM KV cache capacity planning: calculate architecture-derived bytes per token, reconcile startup allocation, and canary context, concurrency, FP8 and preemption controls.
October 5, 2026• 25 min read
vllm kv cache capacity planningvllm kv cache memory calculationvllm concurrency
LLM Training Memory Calculator for ZeRO and FSDP
A 2026 planning model for per-rank LLM training memory, ZeRO and FSDP sharding, GPU count, CPU or NVMe offload, topology rounding, and measured validation.
September 18, 2026• 21 min read
llm training memory calculatorzero memory calculatorfsdp memory calculator