inference optimization

3 articles

ROCm Hyperloom 1.0: Inference Optimization Pilot Gates

A 2026 enterprise guide to ROCm Hyperloom 1.0 inference optimization: compatibility, credential and egress controls, independent correctness checks, benchmark acceptance and rollback evidence.

October 23, 2026• 23 min read
rocm hyperloomrocm hyperloom 1.0inference optimization

Triton Dynamic Batching Tuning: SLO and Rollback Runbook

A 2026 runbook for Triton dynamic batching tuning: verify the baseline, benchmark realistic arrivals with Perf Analyzer, budget queue delay, and apply explicit SLO acceptance and rollback gates.

October 16, 2026• 23 min read
triton dynamic batching tuningtriton dynamic batchingtriton inference server

vLLM KV Cache Capacity Planning: Context and Concurrency

A 2026 engineering guide to vLLM KV cache capacity planning: calculate architecture-derived bytes per token, reconcile startup allocation, and canary context, concurrency, FP8 and preemption controls.

October 5, 2026• 25 min read
vllm kv cache capacity planningvllm kv cache memory calculationvllm concurrency