chunked prefill vllm

1 article

Chunked Prefill Tuning: Token Budget, TTFT and ITL

How max_num_batched_tokens and chunked-prefill-size trade TTFT against ITL in vLLM and SGLang, with defaults, worked arithmetic and a sweep protocol.

October 28, 2026• 15 min read
chunked prefill vllmmax_num_batched_tokens tuningchunked prefill ttft itl tradeoff