gpu inference
5 articles
Kubernetes 1.37 Memory QoS: GPU Inference Tradeoffs
A 2026 operator guide to Kubernetes 1.37 Memory QoS for GPU inference: inactive defaults, host DRAM protection, page-cache risks, reproducible canaries, SLO evidence, and rollback checks.
NVFP4 vs MXFP4 vs FP8 Inference Compatibility Guide
A 2026 compatibility guide to NVFP4, MXFP4, and FP8 across vLLM, SGLang, TensorRT-LLM, NVIDIA and AMD GPUs, with native versus emulated paths, artifact checks, and a reproducible validation protocol.
CPU vs GPU LLM Inference: Low-Concurrency Cost Calculator
A 2026 guide to private CPU vs GPU LLM inference costs: matched-model tests, p95 latency objectives, whole-server energy, and a trace-driven calculator for reuse, purchases and fleet growth.
MIG vs MPS vs Time-Slicing: GPU Cost and Isolation
A 2026 engineering comparison of exclusive GPUs, MIG, MPS and time-slicing for inference, with isolation guarantees, model-fit profiles, SLO tests and cost per qualified tenant.
AWS G6f Fractional vs Whole L4 Break-Even Analysis
A 2026 cost and capacity plan for twelve small inference endpoints, comparing AWS G6f fractional profiles, whole-L4 G6 packing, CPU controls, replicas, memory headroom, and SLO-compliant cost.