kv cache calculator
2 articles

MHA vs GQA vs MLA: How Attention Design Sets KV Cache Size
Per-token KV cache bytes for MHA, GQA and MLA from model config.json: Llama 2 7B, Llama 3.1 8B/70B, DeepSeek-V3 at BF16 and FP8, scaled to 128K and batch.
October 28, 2026• 14 min read
kv cache size calculationkv cache memorykv cache calculator

vLLM KV Cache Capacity Planning: Context and Concurrency
A 2026 engineering guide to vLLM KV cache capacity planning: calculate architecture-derived bytes per token, reconcile startup allocation, and canary context, concurrency, FP8 and preemption controls.
October 5, 2026• 25 min read
vllm kv cache capacity planningvllm kv cache memory calculationvllm concurrency