inference benchmarking
5 articles
ROCm Hyperloom 1.0: Inference Optimization Pilot Gates
A 2026 enterprise guide to ROCm Hyperloom 1.0 inference optimization: compatibility, credential and egress controls, independent correctness checks, benchmark acceptance and rollback evidence.
GenAI-Perf to AIPerf Migration: Preserve Baselines
A 2026 guide to GenAI-Perf to AIPerf migration: reconcile latency and token metrics, qualify client capacity, preserve benchmark artifacts, and define reproducible SLO acceptance gates.
CPU vs GPU LLM Inference: Low-Concurrency Cost Calculator
A 2026 guide to private CPU vs GPU LLM inference costs: matched-model tests, p95 latency objectives, whole-server energy, and a trace-driven calculator for reuse, purchases and fleet growth.
LLM Inference Metrics: TTFT, ITL, TPS and Goodput
A 2026 worksheet for comparing LLM inference benchmarks: define TTFT, ITL, per-user and aggregate TPS, normalize workload and timing boundaries, and reject unsupported goodput claims.
DeepSeek R1 Self-Hosting Cost per Million Tokens (2026 Guide)
Analyzes DeepSeek R1 self-hosting costs per million tokens in 2026: GPU hardware requirements (8x H200 to 16x H100), benchmarked $0.11 to $16 per million token ranges, API pricing, and real deployment case studies.