llm inference
5 articles
LLM Inference Roofline Model: FLOPS vs HBM Worksheet
A 2026 buyer worksheet for LLM inference roofline analysis: calculate arithmetic intensity, check precision-specific GPU peaks, separate prefill from decode, and request evidence for an SLO-qualified pilot.
KV Cache Memory Calculator: Context, Concurrency and GPUs
A 2026 guide to architecture-aware KV cache sizing: calculate bytes per token, replay prompt/output distributions, reconcile vLLM and TensorRT-LLM capacity, and validate replica count against latency targets.
Disaggregated Prefill/Decode Private Fleet Break-Even
A 2026 break-even model for private disaggregated prefill/decode fleets, covering SLO goodput, KV transfer, idle pool-hours, HBM opportunity cost, and conditional routing.
AWS Inferentia2 vs NVIDIA L4: True Migration Cost
A 2026 buyer's guide to Inf2 versus G6/L4, covering pinned-stack feasibility, successful-token cost, migration ledgers, and measured payback.
vLLM 0.29 Upgrade Guide: Runner V2 and API Checks
A 2026 production upgrade guide for vLLM 0.29.0 covering Model Runner V2 validation, immutable CUDA and ROCm artifacts, endpoint exposure, canary tests, and rollback evidence.