ray serve

1 article

Disaggregated Prefill/Decode Private Fleet Break-Even

Disaggregated Prefill/Decode Private Fleet Break-Even

A 2026 break-even model for private disaggregated prefill/decode fleets, covering SLO goodput, KV transfer, idle pool-hours, HBM opportunity cost, and conditional routing.

9/22/202622 min read
disaggregated prefill decodeprefill decode disaggregationllm inference