Prefix Cache Reuse Audit: Tokens, Eviction, Tenant Limits

Executive Summary

A prefix cache reuse audit answers one operational question: when a prompt that looks repeated does not reuse earlier work, is the cause the tokens, the tenant policy, the engine configuration, or retained-cache pressure? This report proposes a single-replica, request-trace method for answering it before any decision to buy GPUs or adopt an external cache. As of October 31, 2026, the vLLM documentation defines block identity as a hash of the block's tokens plus the tokens before it, with extra identity such as LoRA (low-rank adapter) IDs, multimodal hashes and cache salts. Only full blocks are cached, and reuse reduces prefill, not decode.

The central finding is structural: a miss is a statement about the first token position where two requests diverge, not about how similar the prompts look. A timestamp in the first 16 tokens can zero out reuse for a 1,200-token prompt even when roughly 988 of 1,200 tokens are identical, and the audit's worksheet makes that visible with an eligibility ceiling of floor(shared prefix tokens / block tokens) x block tokens, further capped because the last prompt token is always recomputed ^[1]. The report defines eight miss classes, each with a falsifying paired test, and a named "unknown" class so attribution is never forced.

Counters need careful handling. vLLM exports prefix cache queries and hits in tokens, not requests, external hits are counted separately, and counters reset to zero when the instance restarts ^[2]. A token hit ratio is therefore the delta of hits over the delta of queried tokens in one stable window, and it is undefined when the denominator is zero. Averaging per-replica ratios is wrong: a hypothetical 60% replica on 1,000 tokens and a 20% replica on 9,000 tokens yield 24%, not 40%.

Tenant isolation is a constraint, not a defect. The cache salt is injected into the first block's hash so only same-salt requests share blocks ^[3], and academic work describes timing channels when caches are shared across users ^[4]. Per-user isolation prevents cross-tenant leakage but eliminates shared reuse ^[5], so the audit never tries to raise reuse across approved trust boundaries.

Existing guides already explain prompt layout ^[6] and distributed routing ^[7]; recent research proposes trace-based working-set estimation ^[8]. No benchmark was executed for this report, so it publishes a worksheet and illustrative arithmetic, never a claimed speedup. A linear saved-prefill fraction is not a throughput or GPU-count forecast. The decision order is: preserve required prompt semantics, stabilize representation where valid, correct configuration, change retention policy, and only then hand evidence to routing and capacity references.

Introduction and Background

Operators of private inference services often notice that prefix cache hit rate is lower than the prompt design suggests. A system prompt is shared, a document set is shared, a conversation grows turn by turn, and yet the engine reports a modest fraction of reused tokens. The tempting responses are to add GPUs, to bolt on an external cache tier, or to disable tenant separation. Each of those is expensive or risky before the cause is known.

This report is written for enterprise private-inference operators, application platform engineers and performance owners. It treats the question as an audit: capture what the engine actually saw, classify each miss against a closed taxonomy, and run controlled paired tests that can falsify each class. It deliberately starts with one pinned serving replica, so that multi-replica routing is a confounder to isolate rather than the subject. The author, GPU Smith, is an independent private AI infrastructure engineering consultancy acting here as an adjacent advisor, not a provider of inference engines, caches or cloud capacity.

Useful guidance already exists, and this report does not claim otherwise. The llm-d project describes distributed cache locality and scheduling, noting that each vLLM pod manages its own cache in isolation. The Modular handbook explains exact-prefix mechanics and prompt organization, including that even a single character difference breaks the cache. A Ceph engineering report documents a reproducible cold-start test method for external caching ^[9]. A September 2026 preprint, KVSET, defines a KV (key-value) cache working set as the minimum capacity needed to reach a target hit rate ^[8]. The distinction here is narrower: a single-replica trace that separates changed token identity, deliberate policy separation, block eligibility, cold state and eviction, and that ends in a decision about where to act.

Two scope notes apply. First, vLLM documentation moves quickly; the pages read for this report are the "latest" documentation as of October 31, 2026, and PyPI listed vllm 0.32.0 with an October 18, 2026 upload date ^[10]. Operators should pin the exact version, model, attention layout and tokenizer revision they run, and re-read the matching documentation before copying any command. Second, the mechanics below are drawn from documentation and source comments, not from a benchmark run by GPU Smith. They motivate tests; they do not replace them.

Key Changes in the Reuse Contract

This section collects the mechanics that decide whether a repeated prompt can reuse earlier work. Each is stated as documented, with the audit implication following.

Block-chained token identity

vLLM hashes each KV block by the tokens in the block and the tokens in the prefix before it ^[11]. The hash combines a parent hash, the block's exact tokens and extra values that make the block unique, such as LoRA IDs ^[11]. Because each block depends on every earlier block, a single changed token early in the prompt changes every later block's identity. For the first block, a seed stands in for the missing parent, and the source describes it as the hash seed for the first block of any prefix block sequence ^[3].

The audit implication is that identity is token identity, not text identity. Two prompts that render to the same words can still differ if the chat template, special tokens or tokenization boundary differ. The Hugging Face documentation notes that chat templates already include the necessary special tokens ^[12], so a client that adds them again changes the token sequence. The Tokenizers documentation describes pre-tokenization as splitting input so that the model does not build tokens across multiple splits ^[13], and the Transformers tokenizer summary notes that a pre-tokenizer splits text on whitespace or other rules ^[14], and the SentencePiece README guarantees identical tokenization from the same model file in any environment ^[15], which is why the tokenizer revision belongs in the trace. A research paper names the prompt boundary problem, in which tokenizing a prompt and a completion separately differs from tokenizing the combined text ^[16].

Hash algorithm choice is a configuration fact worth recording. On recent vLLM releases the default hashing algorithm is sha256 (confirm it on your pinned version), and the engine arguments page explains that it is the default because it is the most secure choice to avoid hash collisions ^[17]. For a single-replica audit this mostly matters for restarts and upgrades: record the algorithm and version, because comparing fingerprints across versions or algorithms is not meaningful without them.

Full-block eligibility and the last-token rule

Only full blocks are cached ^[11]. The source documents a second limit: when all tokens would hit the cache, the engine must still recompute the last token to obtain logits ^[1], and because allocation is block aligned this can trigger recomputation of an entire block ^[1]. So even an identical request sent twice cannot report a 100% token hit.

The default block size is 16 tokens according to the PagedAttention paper ^[18], but current engine documentation describes the setting only as a block size in tokens, so record the effective value from your startup log rather than assuming it.

Hybrid-model group constraints

The conventional formula assumes one full-attention cache. Hybrid models differ. In vLLM's hybrid KV cache manager, the cache hit prefix of each group is checked and the intersection is returned ^[19]. For Mamba-style state, the documentation says state is stored only on the Mamba block grid ^[20], so a hit can resume only at a block boundary of that grid. For these models, the eligibility ceiling in this report is only a starting hypothesis and must be replaced by the documented group constraints for the pinned version.

Tenant separation with cache_salt

vLLM supports isolating prefix cache reuse through an optional per-request salt. The value is injected into the hash of the first block, which means only requests with the same salt can reuse cached blocks ^[11]. The documentation frames the purpose as preventing an adversary from inferring cached content by observing latency differences, and the server documentation describes the request field as a guard against guessing prompts in multi-user environments. The source comment adds that the salt remains the mechanism for intentional cache isolation ^[3].

Adapters carry identity too. The source includes the adapter path in the hash so that re-pointing a LoRA name at a different adapter changes block identity ^[3]. In an audit, the tenant and adapter label of every request is therefore a column in the worksheet, represented by an approved opaque label and never by the salt value itself.

Counters, per-request evidence and resets

The production metrics define prefix cache queries in terms of queried tokens ^[21] and hits in terms of cached tokens ^[21]. External hits from a KV connector are counted in separate metrics ^[21], and a by-source counter splits hit tokens by the tier that supplied them ^[22]. The metrics design page explains that the periodic log line uses a window fixed to the 1,000 most recent queries ^[23], so it is not interchangeable with a counter delta over your own window.

For per-request evidence, the server can include prompt token details in the usage object when started with the corresponding flag, which defaults to off ^[24]. The statistics module separates tokens prefilled locally as actual compute work ^[25] from cache-hit tokens, and a histogram reports new KV tokens computed during prefill. These are the fields that let a trace row say how many tokens were actually processed.

Three bookkeeping rules affect reconciliation. Requests that skip the lookup, for example when prompt logprobs are required, do not use the cache ^[1], and the V1 guide states that such requests ignore the prefix cache and recompute the prefill of the full prompt. Stats are recorded at admission, so unscheduled lookups are not counted. Previously preempted requests are tracked in separate fields ^[25], and per-request prompt statistics are tracked on the first scheduled prefill rather than on post-preemption repeats ^[26]. Whether your pinned version exports recomputation after preemption in the main counters should be verified, not assumed.

Request-Trace Audit: Capture and Miss Taxonomy

What to capture

The audit begins by writing down the qualified workload: one model, one tokenizer and chat template revision, one replica, one tenant policy, one cache mode, and the user latency and quality requirements the change must respect. Then capture, per request, the input the engine actually saw. Do not retain private prompt bodies in any shared or public artifact; keep fingerprints and counts.

• Rendered message bytes: the exact string after the chat template, hashed with a keyed or salted digest of your own, not stored in clear text.
• Token IDs: the sequence after tokenization, or at minimum fingerprints at fixed checkpoints (for example the first 16, 64, 256 and 1,024 tokens).
• Template inputs: system and tool templates, tool definition ordering, document ordering, and any timestamp, request ID or user field placed in the prompt.
• Revisions: model, tokenizer, chat template and adapter revisions, plus the vLLM version and hash algorithm.
• Policy label: an approved opaque tenant or trust-group label standing in for the salt value.
• Timing and placement: arrival time, serving instance, queue delay and TTFT (time to first token).
• Engine evidence: cached tokens from the usage object where enabled, and metrics deltas around the request.

A caution on the cached-token field: it is a hit signal. Research on shared caches shows that response timing depends on what has been cached, so cached prompts are processed faster than non-cached prompts. Keep per-request cached-token counts in internal telemetry. Whether to expose them to tenants is a policy decision for the operator; this report's inference is that mutually distrustful tenants should not receive them.

The miss taxonomy

Table 1 below lists the miss classes. Each pairs an observable signal with a falsifying test and a next action. Classes are mutually exclusive per request at the first divergence, and "U" is mandatory when evidence is missing.

The table is deliberately conservative. M4 is not a miss to repair, because separation by salt is the documented purpose of that field. M5 is distinguished from M0 by timing and pressure, since vLLM evicts the least recently used free block ^[11] and a request's last block is evicted first because it hashes more tokens. M6 is included only so that it can be excluded on a pinned replica. M2 is common in retrieval workloads, where the llm-d post notes that the exact documents and their order often change between queries ^[7].

The token-identity worksheet

Table 2 shows the worksheet columns with one illustrative row. The row is a hypothetical example, not a measurement.

The eligibility ceiling for conventional fixed-block, full-attention caching can be written as floor(min(shared prefix tokens, prompt tokens minus 1) / block tokens) x block tokens. This is a derived planning formula based on the documented full-block rule and last-token recompute rule, not a published vLLM expression, and it applies to conventional fixed-block full-attention caching only. If the request or cache contract caps reuse lower, use the lower cap. Hybrid models need their documented group constraints instead.

Interpreting the row: 1,000 shared initial tokens with a 16-token block gives floor(1000 / 16) x 16 = 992 eligible tokens, and an observed reuse of 992 is explained, so the class is M3, not a defect. A gap between ceiling and observation is what demands further classification.

Controlled Paired Tests and Counter Reconciliation

Paired tests with a named baseline

Each experiment changes one factor and restores a named baseline before the next. The Ceph report offers a pattern worth copying: it restarted the engine and cleared stored blocks so that all KV caches were cold at the beginning of each test ^[9]. For a single replica, the equivalent is a restart or a documented cache reset. vLLM exposes a reset endpoint in development mode, listed as able to disrupt service, and the documentation warns that the development endpoints should not be used in production ^[27]. A reset removes all cached hashes so that no new blocks will hit ^[28], but it fails if blocks are still in use ^[28]. Use these on a staging replica, drain the queue first, and check the returned status.

The vLLM benchmarking documentation adds a warning that applies to audits too: repeating a run against the same server can reuse prompts left in the prefix cache and inflate results. That is why baseline restoration is a named step, not an afterthought.

Table 3 lists the paired tests. All use synthetic, approved prompts.

Table 3 contains two points that often surprise teams. First, T6 does not measure a time-to-live. Hosted APIs document lifetimes, for example a default 5-minute cache lifetime that is refreshed on use ^[29] with a 1-hour option ^[29], or, under OpenAI's model-dependent retention options, typically around 5 to 10 minutes of inactivity for the in_memory option, with longer retention options available ^[30], but in vLLM retention follows free-queue order and memory pressure. The vLLM design page describes eviction from the head of the free queue, so a long gap on an idle server may still hit, and a short gap on a busy one may not. Second, T5 depends on the ingress. The salt is a request field, so a gateway that drops or rewrites unknown fields can silently place every tenant in the default group; the test must run through the real ingress path, and a pass means labels A and B did not share while two requests with label A did.

Hosted-provider documentation echoes the exact-prefix rule and is useful context for application teams. One provider states that cache reuse requires the entire rendered prefix to match ^[31], another that changes at each level invalidate that level and all subsequent levels ^[29], and a third advises putting large, common content at the beginning of the prompt (Source: ai.google.dev). Those are each vendor's own documented behavior for its own service, and they are consistent with the engine-level mechanics above without being evidence about any self-hosted deployment.

Reconciling counters with per-request evidence

Reconciliation compares two ledgers: the sum of per-request cached tokens and queried tokens in the trace, and the delta of exported counters over the same window. The rules are:

• Use tokens, not requests. Prefix queries and hits are token counters, so a request hit rate is a different quantity.
• Use deltas within a stable process. The token hit ratio is delta hits divided by delta queried tokens over a window with no restart; it is undefined when the delta of queried tokens is zero.
• Never average ratios. Weight by queried tokens; an unweighted mean of replica ratios is wrong.
• Mark reset boundaries. Counters reset to zero when the instance restarts ^[23]; Prometheus counters can only increase or reset to zero on restart ^[2], and rate functions adjust for such resets ^[32]. Take the rate before aggregating so resets are detectable ^[32], and remember that range functions extrapolate to the ends of the window ^[32], so exact reconciliation should use two direct scrapes.
• Separate local from external. External hits are separate metrics, and the by-source counter shows which tier supplied hit tokens.
• Do not count retries as unique demand. Deduplicate by an idempotency key in the trace; a retry of an identical request inflates queries and hits together.
• Distinguish the log line from counters. The periodic log hit rate uses a fixed recent-query window and the in-process window is cleared on reset, so use counter deltas, not the log line, for the audit.

The exported metric names may carry a counter suffix on the scrape endpoint; confirm names with a direct scrape on the pinned version before writing queries. A query of the form increase(hits)[window] divided by increase(queries)[window], guarded by a condition that the queries delta is greater than zero, implements the rule. Where the guard fails, record "undefined", not zero; in PromQL, series with no matching entry in the opposite vector are not part of a division result ^[33], so an empty window shows up as a missing sample, not a zero.

Implementation Considerations and Process Changes

Decision order

Once classes are attributed, the action follows the largest attributed share of queried tokens. The order matters because each earlier step is cheaper and less risky than the next.

• Preserve required semantics first. If a changing field is functionally required at the start (for example a policy banner that must precede other text), do not move it; record the cost.
• Stabilize representation where valid (M1, M2). Move timestamps, request IDs and user-specific content after the shared material, which is the layout advice in the Modular handbook ^[6] and in hosted-provider guides that say to put stable instructions and shared reference material first ^[31]. Normalize serialization: unstable key ordering and structure causes misses for logically identical content ^[6], and a hit also depends on serializing previous messages identically ^[6]. Fix document ordering where ranking is not semantically required, and keep tool definitions in a stable order, since hosted-provider guides list tool names, descriptions, schemas and ordering among the changes that alter the rendered prefix ^[31].
• Correct configuration (M7). Confirm caching is enabled, that requests are not asking for features that skip the lookup, and that the hash algorithm and block size are what you recorded. The Modular handbook notes that disabling prefix caching is usually not a performance optimization ^[6], so a disabled flag in an old launch script is a plausible finding; other engines expose equivalent switches, for example SGLang's option to disable RadixAttention prefix caching ^[34].
• Change retention policy (M5). Only after M1 to M4 are excluded. Levers include free KV memory, concurrency limits and request mix. The vLLM optimization guide says preempted requests are recomputed ^[35], that recompute is the V1 default preemption mode, and that preemption and recomputation can adversely affect end-to-end latency. A staging test can force pressure by raising concurrent long requests (T7).
• Pass evidence onward (M6). If the trace shows requests landing on other instances, hand the table to the routing reference. Do not treat routing as the audit's product-selection subject.

Eviction behavior differs by engine, so record it. SGLang's RadixAttention paper describes an LRU policy that evicts the least recently used leaves ^[36], marks a node evictable only when its reference counter is zero, and lets cached tokens and running requests share one memory pool. SGLang's server arguments expose the radix tree eviction policy as a setting ^[34].

Tenant boundaries stay intact

The audit never recommends removing salts to raise hit rate. Hosted services draw similar boundaries; one provider documents that caches are not shared across organizations ^[31]. Research on timing in shared caches distinguishes sharing scopes: per-user caching has no potential privacy leakage from prompt caching ^[4], while global sharing carries the highest risk ^[4], and that paper concludes only per-user caching should be allowed where leakage matters ^[4]. A later paper describes an adaptive black-box adversary who can create arbitrary accounts ^[5], and notes that per-user isolation prevents cross-tenant leakage but eliminates shared reuse ^[5]. Another work chose prefill time as the attack vector because the attacker can stop after the first token ^[37], and lists distinct cache namespaces as a mitigation.

The practical rules for the audit are: run probes with the same labels production uses; never run hit-versus-miss timing probes across tenants on a production replica; and treat M4 rows as correct behavior. The detection method in the research uses a one-sided statistical test, because hits are expected to be faster ^[4], with randomized sample order; an operator checking that salts truly separate groups in staging can borrow that design, using synthetic prompts. Because miss timing varies from request to request, a single slow or fast request proves nothing. The threat model assumes an ordinary user with no direct control over the inference server ^[38].

Validate before promoting

A change is promoted only after a before and after record is complete. Publish a blank worksheet in the runbook and fill, for each candidate change, the following columns:

• Actual processed input: tokens computed in prefill, from the computed-tokens histogram ^[25].
• Cache-reused tokens: local and external, kept separate.
• Queue delay: the waiting-phase histogram.
• TTFT and inter-token latency distributions: percentiles, not means.
• Successful output: completion rate and goodput, meaning requests meeting the latency objective.
• Semantic quality: an agreed evaluation set, because moving static text can change model behavior.
• Tenant boundary check: T5 results through the real ingress.
• Rollback condition: the previous template and configuration, and the trigger for returning to them.

Semantic quality deserves emphasis. Reordering a prompt to create a longer static prefix changes what the model reads first. A reuse gain that lowers answer quality is not a gain, and the audit treats it as a failed candidate.

Data Analysis and Evidence

This section separates three kinds of numbers: published figures from other parties with their stated setups, documented defaults, and the hypothetical arithmetic used to teach the worksheet. None of them is a GPU Smith measurement.

Published figures and their limits

The SGLang paper reports cache hit rates ranging from 50% to 99% across its benchmark suite ^[36], and in a month of production use with one worker per model it observed a 52.4% hit rate for one multimodal model and 74.1% for another ^[36]. It attributes hits to common system messages, frequently reused example images and multi-turn histories. These figures show that hit rate is a property of workload shape, which is why this report avoids a target number. The llm-d post reports a benchmark on a cluster with 16 H100 GPUs in which the precise-scheduling configuration delivered a P90 TTFT of 0.542 seconds against more than 90 seconds for cache-blind schedulers (^[7]). That is a project-reported multi-replica routing result; it addresses the confounder this audit isolates, not the single-replica attribution question.

The research literature adds one caution on sizing. KVSET estimates working sets using the Mattson stack algorithm to estimate hit rates across capacities ^[8], and reports validation on production LLM traces ^[8]. It is a submitted preprint of September 23, 2026; this report cites it as research context, not as a tested tool, and not as an algorithm invented here.

On external caches, the LMCache documentation describes turning KV cache from temporary state into reusable state ^[39], and its calculator estimates how many tokens fit in available GPU memory ^[40]. The Ceph report notes that bytes per token is model dependent ^[9] and states that KV caching can reduce TTFT more than tensor parallelism in its cold-start tests on its own hardware ^[9], a claim about an external tier, not about the single-replica attribution problem. SGLang's hierarchical cache builds on its radix tree ^[41], and LMCache documents non-prefix KV reuse ^[39]; both belong to tiering and capacity discussions. Those are capacity and tier questions, handled by the capacity and tiering references, and are out of scope here.

Illustrative arithmetic (Hypothetical Example)

Assume a 16-token block (verify on your system), a request of 1,200 tokens made of a 1,000-token static prefix and a 200-token user suffix. All numbers are illustrative.

• Same request twice: maximum hit length is capped at 1,199 tokens, so the ceiling is floor(1199 / 16) x 16 = 1,184 tokens, or 98.7% of 1,200.
• Static prefix, changing suffix: shared tokens are 1,000, so the ceiling is floor(1000 / 16) x 16 = 992 tokens (62 blocks), or 82.7%.
• Timestamp at token 12: the longest common initial run is 12 tokens, so floor(12 / 16) x 16 = 0. Roughly 988 identical tokens after the timestamp earn nothing, because every later block's hash chains from the changed block. This is M1.
• Timestamp moved to token 1,150: floor(1150 / 16) x 16 = 1,136 tokens (71 blocks), or 94.7%.
• Two 400-token documents swapped after a 200-token system prompt: the shared initial run is 200 tokens, so floor(200 / 16) x 16 = 192 tokens, or 16.0%. This is M2.

Window arithmetic follows the same discipline. Suppose a counter scrape at the start of a window shows 10,000 queried tokens and 4,000 hit tokens, and a later scrape in the same process shows 22,000 and 9,400. The deltas are 12,000 queried and 5,400 hit tokens, so the token hit ratio is 5,400 / 12,000 = 45.0%. Now suppose two replicas report ratios of 60% on 1,000 queried tokens and 20% on 9,000. The unweighted mean is 40%, but the weighted ratio is (600 + 1,800) / 10,000 = 24%. If the queried delta had been zero, the ratio would be undefined, not zero.

The saved-prefill fraction deserves a final warning. If 992 of 1,200 prompt tokens are reused, the linear saved-prefill fraction is 82.7%, but reuse reduces prefill work only ^[20] and does not reduce the time of generating new tokens ^[20]. A request dominated by a long output sees little benefit. The fraction is therefore not a throughput or GPU-count forecast, and queueing, batching and decode length all intervene.

Implications and Future Directions

For operators, the immediate implication is procedural. A hit-rate dashboard without a miss taxonomy invites the wrong spending decision, and a taxonomy without paired tests invites rationalization. The worksheet turns "hit rate is low" into a ranked list of token-level causes, each with a falsifier. A team that finds most queried tokens in M1 and M2 has a serialization project, not a hardware project. A team that finds M4 dominant has a correctly isolated multi-tenant service and should look at per-tenant prefix design instead. A team that finds M5 dominant has evidence for a retention discussion, which may eventually lead to capacity or tiering decisions, supported by working-set analysis of the kind KVSET proposes.

Several areas remain open and are flagged as unverified here. Hybrid and state-space models have documented group constraints that change the eligibility ceiling. Feature combinations can also change what a shared text prefix means for cached state. Counter semantics around preemption and reset should be re-verified at each upgrade, since the sources show the statistics code tracks preempted requests separately (^[25]). The effective model and attention cache path must be confirmed for the pinned release, because documentation pages for one version can disagree with each other on model support.

For security reviewers, the audit supplies evidence about whether policy separation behaves as documented without weakening it. Any proposal to share prefix state across trust boundaries remains a separate risk decision, informed by the timing research summarized above.

Frequently Asked Questions (FAQs)

What is prefix caching in LLM inference, and what does it save?

Prefix caching stores KV blocks from processed requests and reuses them when a new request begins with the same tokens. The vLLM design documentation describes it as avoiding redundant prompt computations. It reduces prefill time, not decode time.

How does KV cache prefix reuse decide a match?

A block matches only if its tokens and every earlier block's tokens match, plus matching extra identity such as adapter and salt. Partial trailing blocks are not cached.

How is a prefix cache hit rate audited?

Capture token IDs and fingerprints per request, compute the longest common initial token count against a reference, derive the eligibility ceiling, compare with observed reuse, and assign a miss class from Table 1. Reconcile with counter deltas over a stable window.

What is the token identity cache key in vLLM?

Conceptually it is the chained hash of parent hash, block tokens and extra hashes. The algorithm defaults to sha256 on recent releases; confirm it on your pinned version.

Which eviction policy applies?

For vLLM's block pool, the least recently used free block is evicted first. SGLang describes LRU eviction of radix-tree leaves with a configurable policy setting.

Does multi-tenant prefix cache isolation cost reuse?

Yes, by design. Same-salt requests share and others do not. Research states that per-user isolation eliminates shared reuse across users, as described above.

Can timing reveal another tenant's prompt?

Research shows that when a cache is shared across users, response time depends on what has been cached, which is why salts are documented as a timing mitigation. Shared-cache designs across trust boundaries need a security decision, not a hit-rate target.

Why would a long idle gap still hit, or a short gap miss?

Retention in vLLM follows free-queue order and memory pressure, not a fixed lifetime as in hosted APIs ^[29].

Conclusion

Low prefix reuse is a diagnosis problem before it is a capacity problem. The documented mechanics show that identity is token identity chained across blocks, eligibility is block-aligned, tenant labels deliberately split the cache, and counters are token-weighted with defined reset behavior. From those, this report builds a worksheet with eight miss classes plus an explicit unknown, controlled paired tests that can falsify each, and a reconciliation procedure that avoids averaged ratios and zero denominators.

The decision order is to preserve required semantics, stabilize representation where valid, correct configuration, change retention policy, and only then pass evidence to routing and capacity references. Tenant boundaries stay intact throughout: rows classified as deliberate policy separation are treated as correct behavior, and no step in the method asks for reuse across a trust boundary. Teams that act on the table in order will usually find that the cheapest causes, unstable serialization and early variable fields, are exhausted before a retention or capacity discussion is warranted. No benchmark was executed for this report; the arithmetic is illustrative and the sources are documentation, source comments and published research, each dated or versioned where the source permits. Operators should pin their versions, record the block size, hash algorithm and tokenizer revision, re-read the matching documentation, run the paired tests on a staging replica with approved synthetic prompts, and promote changes only after latency, quality, tenant-boundary and rollback checks are recorded.