llm inference cost
4 articles
Speculative Decoding Economics: When Draft Models Cut Cost
Speculative decoding economics for private LLM fleets: acceptance-rate math, draft memory cost, concurrency limits and a 2026 worksheet for cost per million accepted tokens.
FP16 vs FP8 vs INT4 Inference Cost: Quality and GPU Fit
A 2026 guide to quality-adjusted GPU inference cost, comparing FP16/BF16, FP8 and exact INT4/FP4 recipes with a compatibility crosswalk, integer GPU-fit examples, acceptance gates and a Python calculator.
CPU vs GPU LLM Inference: Low-Concurrency Cost Calculator
A 2026 guide to private CPU vs GPU LLM inference costs: matched-model tests, p95 latency objectives, whole-server energy, and a trace-driven calculator for reuse, purchases and fleet growth.
Kimi K2 Hardware Requirements: VRAM, Nodes and Cost to Run
A 2026 guide to Kimi K2 hardware requirements covering VRAM and GPU needs, quantization tiers from 245GB to 1.09TB, Moonshot's 16-GPU node configuration, and self-hosting versus API cost.