GenAI-Perf to AIPerf Migration: Preserve Baselines

Executive Summary

GenAI-Perf to AIPerf migration should be treated as a measurement change with a compatibility bridge. NVIDIA's September 18, 2026 walkthrough identifies AIPerf as the ground-up successor to GenAI-Perf. ^[1] The migration guide also describes compatibility for currently supported features; these statements concern different things, architecture and supported interfaces. ^[2] As verified on October 13, 2026, the latest stable package is AIPerf 0.13.0, published September 24; subsequent development builds should not silently enter an acceptance test. ^[3] The practical sequence is to freeze the legacy evidence, qualify the new client, run both against an unchanged deployment, and authorize a new baseline only after reconciling the measurement definitions.

The largest semantic traps concern reasoning output and latency populations. When the endpoint returns reasoning separately in reasoning_content and the selected adapters handle that field as documented, compare legacy time to first token (TTFT) with AIPerf time to first output token (TTFO), and legacy output sequence length (OSL) with AIPerf Output Token Count. ^{[4] [5]} Reasoning embedded in ordinary content, such as <think> blocks, can remain in output counts unless filtered and may affect first-output timing. ^[6] Before applying this crosswalk, inspect a representative raw response or stream and verify each adapter's parsed fields, token counts and first-output event; model capability alone does not establish separation. Also examine inter-token latency (ITL) separately: the current AIPerf reference describes a token-normalized decode duration, whereas the legacy documentation describes intermediate-response gaps divided by tokens in the latter response. ^{[7] [8]} Streaming chunks can contain multiple buffered tokens, so a shared metric name cannot establish equivalence. ^[9]

Use separate regression, service-load and trace-replay profiles. Preserve rendered prompts, model and tokenizer revisions, actual output lengths, cache procedures, arrival timing, streaming settings and artifact provenance. Tokenizer revisions can identify commits, and container digests pin image content rather than a movable tag. ^{[10] [11]} Increase client workers while holding server configuration constant, then cross-check server counters and client resource pressure; a plateau alone does not prove server saturation. Prometheus rates require reset-aware handling, and Linux pressure measurements expose stalled tasks. ^{[12] [13]}

Acceptance should combine latency percentiles, error accounting, output throughput and SLO-compliant goodput, where SLO means service-level objective. AIPerf's good-request fraction keeps errors in its denominator. ^[14] This report provides a migration crosswalk, artifact manifest, traffic matrix and explicitly hypothetical calculations. Its confidence example uses the Wilson interval, not an invented hardware result. ^[15] Procurement acceptance should require comparable workload evidence and correctness checks alongside speed, consistent with the separation of performance and accuracy in established inference benchmarking. ^[16]

Introduction and Background

Replacing a benchmark executable can alter the meaning of an apparently familiar chart. A procurement decision needs to distinguish changes in the server from changes in the client, dataset and accounting. This report examines AIPerf vs GenAI-Perf for private artificial intelligence (AI) platform engineers, performance teams and technical buyers who need to preserve historical large language model (LLM) inference evidence while adopting a new harness.

The relevant event is NVIDIA's successor walkthrough, rather than evidence that AIPerf first existed on that publication date. The release record separately identifies the current stable release and its focus on Kubernetes execution, speculative-decoding metrics and workload control. ^[17] Sources were fetched during this report's October 13, 2026 research session. Mutable documentation should be treated as a dated reference, while the command example uses versioned documentation and an explicit package pin. ^[17]

The engineering perspective is acceptance against a written specification. GPU Smith describes engagements as using written acceptance criteria; it is an adjacent infrastructure advisor in this comparison, rather than another benchmark product. ^[18] The resulting recommendation is therefore a process for producing reviewable evidence, not a claim that changing tools makes a deployment faster.

Three evidence layers should remain distinguishable: the workload contract, the observed client events, and the serving system's counters. OpenAI's Chat Completions reference describes server-sent streaming events, while Prometheus documents counter-rate calculations. Neither substitutes for the other. ^{[19] [12]} A timestamped client observation measures an experienced response; a server counter measures work recorded by that server. Local short-duration measurements can use a performance counter, whose documented purpose is measuring elapsed intervals at high resolution. ^[20]

A fair comparison also requires stable execution identity. Kubernetes documents that image tags can move while digests are fixed; PyTorch cautions that identical seeds do not guarantee reproducibility across releases or platforms. ^{[21] [22]} Consequently, “same model, same seed” is an incomplete baseline description. The acceptance packet should explain what remained fixed, what changed deliberately, and which historical conclusions remain supportable.

GenAI-Perf: The Baseline to Preserve

Capabilities

GenAI-Perf provides the historical measurement contract that existing capacity and procurement decisions may reference. Its archived documentation defines output token throughput as benchmark output tokens divided by benchmark duration. ^[23] Retain that denominator explicitly. A normalized reporting layer should store numerator, interval and inclusion rules, rather than importing a column with a familiar name into a new dashboard.

The legacy implementation also documents ITL through intermediate responses and their token counts. ^[8] Preserve the raw responses and the implementation version wherever available. A mean, a per-request average and a pooled distribution of response gaps are different statistical objects, even when each is displayed in milliseconds.

Adoption

Before changing the harness, inventory the old baseline with the following controls:

• Execution identity: Capture the installed GenAI-Perf version, associated Perf Analyzer build, client operating environment and serving image digest. Docker documents digest-based image selection. ^[11]
• Model identity: Record weight revision, precision, runtime arguments and tokenizer commit or local directory. Hugging Face supports both revision selection and local vocabulary files. ^{[10] [24]}
• Prompt identity: Save rendered messages and chat templates, not only unrendered text. Models may use different control-token formats. ^[25]
• Sampling identity: Archive the generated corpus and request ordering. GenAI-Perf documents reuse of synthetic payloads during benchmarking. ^[26]
• Randomness: Record the seed explicitly. The legacy documentation says an unspecified seed is randomized. ^[27]
• Warmup and termination: Recover warmup settings, measurement intervals, stabilization rules, request limits and stop reasons. Legacy warmup is separately configurable. ^[28]

When those artifacts are absent, classify the historical row as insufficiently specified. Recover the server digest from image-pull records when available; Docker prints it after pulling. ^[29] Pin the recovered image by digest rather than accepting the current value of its old tag. ^[11] Keep it for context, but establish a reproducible reference before allowing it to gate a hardware purchase.

Strengths and Limitations

The strength of a legacy baseline is continuity: it can tie present measurements to past decisions if its workload and accounting are recoverable. Its limitation is that continuity does not establish equivalence to a new metric definition. The official comparison matrix is a living document, and old stabilization controls cannot simply be assumed to retain their meaning in AIPerf. ^[30]

Repeated execution alone does not repair missing controls. PyTorch notes that algorithm selection can differ between runs because of benchmarking noise and hardware differences. ^[31] Record those runtime choices where relevant, and keep deterministic-operation settings fixed because they can also affect speed. ^[32] The objective is a defensible legacy reference, not an exact reproduction of an undocumented historical number. Controlled reference comparisons follow the same general principle as MLPerf's closed division, which holds the model and reference setup fixed. ^[33]

AIPerf: The Replacement to Qualify

Capabilities

AIPerf uses a scalable multiprocess architecture and an extensible plugin system for endpoints, datasets, transports and metrics. ^{[34] [35]} Python documents subprocess-based multiprocessing as a way to bypass the Global Interpreter Lock. ^[36] That architectural rationale supports testing client capacity; it does not independently prove that a particular client can sustain a target deployment's load.

The versioned dataset reference documents synthetic inputs, custom JSON Lines (JSONL) and public or trace-oriented formats, including ShareGPT, Mooncake and Baseten. ^[37] The repository additionally documents verbatim JSONL payload replay and multi-turn replay of saved inputs. ^{[38] [39]} Preserve the parser choice and the converted dataset checksum, because using the same dataset name is weaker than using identical serialized requests. Include the template's control tokens in this identity check because model formats can differ. ^[25]

Adoption

Adopt the replacement in controlled steps:

• Pin the package: Use the stable version identified in the release record and retain the dependency lock or client container digest. ^{[17] [29]}

• Prepare offline assets: The repository documents local-tokenizer use without Hugging Face access. Preserve those assets with the model package for disconnected testing. ^[40]

• Validate installation: On Linux aarch64, the package documentation says the crick dependency requires a C compiler at installation time. Build that dependency into the approved client image. ^[41]

• Validate the adapter: Test representative serialized requests and parsed responses before generating acceptance load. Chat streaming uses incremental response chunks rather than a single complete response object. ^[19]

• Archive repetition: Keep individual runs and their summaries; the repository documents confidence intervals across repeated runs. ^[42]

• Generation controls: Verify supported stopping controls before enforcing a fixed-length workload. Backend support differs in the vLLM reference. ^[43]

• Dependencies: Freeze runtime dependency versions and algorithm-selection settings with the client image. Identical seeds do not guarantee cross-release reproducibility. ^[22]

• Event accounting: Record whether usage is obtained from the final stream chunk or reconstructed locally. ^[44]

The endpoint families include text generation, embeddings, ranking, audio and image-related workloads. Streaming metrics must be qualified per selected adapter: the CLI disables streaming for endpoint types that do not support it. ^[45] An embeddings result therefore cannot be assigned a meaningful token-stream ITL merely because another endpoint reports one.

Strengths and Limitations

The package's licensing is Apache License 2.0. ^[46] Operationally, the tool's breadth is useful only when the selected adapter preserves the old endpoint semantics. Inspect role handling, system prompts, output limits, reasoning fields, error responses and request controls.

Requested output lengths remain server-dependent. The published repository warns that generation-length constraints require supporting server controls. ^[47] In addition, final streaming usage statistics may be absent after interruption, according to the OpenAI reference. ^[48] Acceptance must therefore verify actual lengths and incomplete-stream handling, rather than treating configured means or missing usage fields as measured facts. Include accuracy checks in the performance run as well as in standalone quality testing. ^[16]

Feature Comparison

The migration crosswalk should be an executable review record: each row names the old interpretation, the proposed replacement and the evidence needed before comparison. Table 1 summarizes the consequential differences. The official guide supplies the reasoning-token mapping; the versioned feature matrix supplies supported-interface boundaries. ^[2]

The table deliberately marks semantic reconciliation as work, rather than claiming every flag substitution preserves a baseline. In particular, the migration guide's broad statement about unsupported analyze functionality is less specific than the current versioned matrix, which describes sweeps folded into profile. ^[30] Use the pinned matrix when porting sweep workflows, and retain a copy with the run packet.

A stronger endpoint check compares rendered messages before comparing tokens. Hugging Face documents conversion of structured messages into a formatted sequence, and warns against duplicating special tokens after template application. ^{[51] [52]} A different template can change the workload even if the visible human text looks identical.

Likewise, an application programming interface (API) usage field is not guaranteed to arrive with every stream event: OpenAI documents final-chunk usage reporting. ^[44] Decide whether the comparison uses server-reported counts or a pinned local tokenizer, record that choice, and reject a mixed-count series. Preserve raw errors and timing records, as the pinned vLLM benchmark reference also supports per-request exports. ^[53]

Performance and Benchmarks

Client saturation must be tested, not inferred from the architecture. Fix the deployment, corpus, arrival policy, cache procedure and client placement, then increase available request workers without changing the workload contract. Record worker placement and execution mechanism; Python distinguishes subprocess-based parallelism from threading. ^[36] Keep concurrency high enough to permit the intended worker comparison; otherwise a worker cap can make nominal settings equivalent.

Use the following observations together:

• Server work: Collect an output-token counter or equivalent server-side work measure and record its exact definition. Prometheus rate reports per-second counter growth. ^[12]
• Counter handling: Calculate rates before aggregation to preserve reset detection, and retain raw scrape timestamps. ^[12]
• Client scheduling: Compare planned dispatch time with actual dispatch time. Python cautions that sleep can last longer than requested because of scheduling activity. ^[54]
• Client resources: Capture host and client-cgroup pressure, not just a single central processing unit (CPU) utilization average. Linux exposes pressure information for cgroups as well as the host. ^[55]
• Transport pressure: Inspect socket queues and memory with ss, and distinguish Transmission Control Protocol (TCP) queues from internal ZeroMQ pipes. ^{[56] [57]}

ZeroMQ's high-water marks describe internal pipe capacity; they are not a ready-made AIPerf health metric. Use exposed logs or instrumentation where available. Missing queue instrumentation should be recorded as an evidence gap, rather than converted into an assertion that queues were healthy. ^[57]

Worker scaling is qualified when additional client capacity no longer changes server throughput beyond a predeclared tolerance, and the timing/resource evidence still shows that the target arrivals were delivered. A server plateau with growing client backlog remains inconclusive. Linux's pressure totals can reveal stalls that short averages obscure, while ss exposes socket statistics and queue columns. ^{[58] [59] [60]}

For latency, record p50, p95 and p99 as chosen acceptance reporting points, but define their populations first. AIPerf distinguishes ITL from the full distribution of consecutive inter-chunk arrival gaps. ^[61] vLLM's documented stream_interval can buffer multiple tokens before sending them. ^[9] Consequently, an event-gap p99 and a per-request decode-average p99 should remain separate series.

Server histogram percentiles also need a separate label: Prometheus interpolates quantiles within buckets. ^[62] They are useful for correlation, but are not numerically interchangeable with percentiles calculated from retained client requests. Preserve bucket definitions when comparing telemetry across runs. ^[62] Capture client and server timing independently before aligning their run windows. Elapsed-time counters do not provide an absolute cross-host timestamp. ^[63] Use a monotonic elapsed-time source for local timing; Python documents that its monotonic clock is unaffected by system-clock updates. ^[64]

No new hardware measurements were conducted for this report. The package's example output is itself labeled as not representing an official benchmark. ^[65] Neither that output nor NVIDIA's architectural explanation establishes an independently verified throughput improvement for a buyer's server, runtime or graphics processing unit (GPU).

Migration and Acceptance Gate

Start with a documented recipe and local smoke test, then perform the bridge on a deployment whose identity is fixed. Kubernetes uses the digest when an image specifies both tag and digest; record the resolved value, not just the human-readable tag. ^[66] Tokenizer loading can use a local vocabulary directory, allowing the same assets to be preserved across both clients. ^[24]

The following regression recipe (Hypothetical Example) uses documentation-checked AIPerf flags. It was not executed against an inference server. Replace MODEL and asset paths with the frozen deployment's values. The selected counts and lengths are illustrative test parameters, not vendor defaults. ^[67]

python -m pip install aiperf==0.13.0
aiperf profile \
  --model MODEL --url http://127.0.0.1:8000 \
  --tokenizer /approved/pinned-tokenizer \
  --endpoint-type chat --streaming \
  --synthetic-input-tokens-mean 512 \
  --synthetic-input-tokens-stddev 0 \
  --output-tokens-mean 128 --output-tokens-stddev 0 \
  --concurrency 8 --request-count 1000 \
  --warmup-request-count 32 --random-seed 42 \
  --workers-max 8 --artifact-dir artifacts/bridge

This generated fixture is useful for a smoke test or a new regression corpus. For the cross-tool bridge, substitute the frozen serialized dataset rather than relying on matching seed numbers. AIPerf documents verbatim payload replay, while GenAI-Perf documents dataset reuse. ^{[38] [26]} Verify the same rendered control tokens and archive request-level responses, errors and timing. ^{[25] [53]}

The command requests an output-length distribution; it does not guarantee achieved OSL. Add server-supported generation controls only after checking payload acceptance. The vLLM benchmark reference illustrates that ignore_eos support differs across backends. ^[43] Also distinguish a harness seed from runtime determinism, which PyTorch does not guarantee across releases. ^[22]

Table 2 defines a recommended manifest. Store secrets separately and retain redacted commands.

The export reference describes independent schema evolution for different output files. Do not infer every schema from the package version. Its telemetry naming description and version table also differ on which schema revision introduced the naming change; inspect the artifact and validate the parser instead of selecting a revision by assumption. ^[70]

Apply explicit pass/fail rules:

• Serialization: Representative rendered requests and parsed responses match the agreed endpoint contract. ^[51]
• Workload: Achieved input and output distributions meet the frozen specification.
• Load delivery: Target arrivals are actually dispatched without unapproved concurrency throttling.
• Client headroom: Worker scaling and resource evidence pass the saturation qualification. ^[13]
• Latency: The agreed p50/p95/p99 thresholds pass using reconciled event and sample definitions.
• Goodput: Joint request-level SLO compliance and error-inclusive attainment pass. Reject missing required metrics and check threshold units before evaluating the gate.
• Correctness: Output quality checks pass alongside speed, following the principle of accuracy verification in performance evaluation. ^[16]
• Reproducibility: A second controlled run meets the declared uncertainty and equivalence rules. ^[42]

Data Analysis and Evidence

A migration has two quantitative questions: whether the client delivered the intended load, and whether the delivered requests met the service contract. Throughput alone answers neither completely. Research such as DistServe defines goodput in relation to SLO attainment, while established inference evaluation makes latency constraints scenario-dependent. ^{[71] [72]}

A concrete published specification illustrates why SLOs must be workload-specific. MLCommons' March 24, 2026 GPT-OSS post describes the DeepSeek-R1 server scenario with Poisson arrivals, a 99th-percentile TTFT limit of 2 seconds and a time per output token (TPOT) limit of 80 milliseconds. ^[73] These are constraints for that benchmark scenario. Enterprise acceptance thresholds should come from the service contract and retain the same metric definitions across the migration.

Table 3 separates recommended acceptance profiles. Here ISL means input sequence length and OSL means output sequence length. These are proposed engineering profiles, not measured benchmarks.

These profiles measure different questions and should retain separate scores. AIPerf's timing reference says concurrency becomes a ceiling for rate-driven requests, while fixed-schedule mode uses dataset timestamps. ^{[74] [75]} Independent workload research also treats arrival processes and length distributions as explicit assumptions. ^[76] Poisson and gamma controls should be stored with the original command, including the gamma smoothness setting. Keep the schedule immutable; vLLM's benchmark documentation similarly describes faithful replay based on timestamps. ^[77]

Client saturation calculation (Hypothetical Example)

Assume worker caps of 2, 4 and 8, with server output rates of 8,000, 8,400 and 8,442 tokens/s. Hold all server settings and workload parameters fixed. Define the relative worker-scaling change as:

delta = 100 * (new_server_rate - previous_server_rate) / previous_server_rate

The assumed changes are 5.00% and 0.50%. If a team predeclares 1% as its plateau tolerance, the last change satisfies that arithmetic criterion. It still needs resource-pressure, socket and dispatch evidence before accepting client sufficiency. Compare wall time with process CPU time carefully: Python's process-time clock excludes sleep. ^[78] Linux pressure exposes resource stalls, and ss provides socket statistics. ^{[68] [59]}

Compute the rates over aligned windows. Prometheus rate extrapolates across imperfect scrape alignment; increase can even return non-integer values for integer counters. ^{[79] [80]} Retain the raw counter series and explain the windowing method rather than rounding away a discrepancy.

SLO goodput calculation (Hypothetical Example)

Assume 1,000 attempted requests during a 60-second measured cohort: 920 satisfy every SLO, 40 complete but miss at least one, and 40 end in error. Successful completion throughput is 960 / 60 = 16.00 requests/s; SLO-compliant goodput is 920 / 60 = 15.33 requests/s. An attempted cohort must retain interrupted streams as outcomes even if final usage is absent. ^[48] The error-inclusive attainment fraction is 920 / 1,000 = 92.00%, rather than the completed-only 95.83%. AIPerf explicitly retains errors in this denominator. ^[14]

With an illustrative normal critical value of 1.96, the Wilson 95% interval for the assumed attainment fraction is approximately 90.15% to 93.53%, using NIST's formula. ^[81] This calculation assumes independent Bernoulli outcomes; a bursty trace may contain correlated requests. NIST also cautions about symmetric normal approximations when samples or failure counts are small. ^[82] Use repeated independent runs or a correlation-aware analysis rather than treating the interval as proof of production reliability. The repository documents repeated-run confidence reporting, but that does not remove dependence within a trace. ^[42]

Apply request-level thresholds jointly, then calculate cohort attainment and rate. An all-threshold request gate differs from separately passing marginal percentiles. The vLLM reference likewise defines request-level goodput metrics for TTFT, TPOT and end-to-end latency. ^[83] DistServe's TPOT is a per-request token-generation average, reinforcing the need to state the sample population before comparing percentiles. ^[84]

Implications and Future Directions

A successful migration separates the new measurement contract from the old operational conclusion. The acceptance owner should declare a baseline epoch, a versioned combination of workload, harness semantics, parser and deployment controls. New epochs make historical charts interpretable without implying that every break is a hardware regression or gain.

Use these decision rules:

• Keep the legacy baseline: Its inputs and accounting are recoverable, and the old accepted system remains unchanged. Preserve it as the historical record.
• Rerun both harnesses: A tool or parser changes while the server remains fixed. Require matched requests and reconciled metrics before linking the series.
• Declare a new epoch: Endpoint behavior, reasoning accounting, tokenizer/template, workload distribution, arrival policy, cache procedure or measurement population changes materially.

The identity controls are practical: Kubernetes documents how digest pinning avoids a registry change producing mixed image versions, and Hugging Face supports revision selection down to a commit. ^{[85] [10]} Conversely, changing deterministic-operation settings can change measured speed, so include that configuration in the epoch. ^[32]

Rollback should restore the last qualified harness image, dataset and reporting parser together. Preserve the original artifact before conversion, and verify that the parser can still reconstruct the previously accepted denominator. Do not silently overwrite historical metrics with renamed fields. Pin restored images by digest so a later registry tag change cannot alter the rollback target. ^[21]

For private deployments, telemetry location matters as much as availability. The legacy documentation places Data Center GPU Manager (DCGM) Exporter on the inference-server machine. ^[86] AIPerf documents DCGM exporter collection and local pynvml GPU queries. ^[87] A local GPU-query backend on a separate client should not be assumed to observe remote server GPUs. Correlate deployment counters, GPU telemetry and client events within a declared measurement window; Python's elapsed-time counters require differences between readings, not cross-host subtraction. ^[63]

Written evidence also supports future upgrades. GPU Smith's stated documentation scope includes architecture records, operating procedures and acceptance reports. ^[88] An effective migration packet applies that principle to benchmark commands, workload files and schema consumers. It should allow another engineer to reproduce the decision, rather than requiring trust in a screenshot. Established inference benchmarking uses a load generator for scheduling and latency tracking, underscoring that the client is part of the measurement system. ^[89]

Conclusion

Preserving GenAI-Perf baselines requires a controlled bridge to AIPerf. Freeze the historical workload and serving deployment, qualify the replacement client, reconcile token and latency definitions, and compare both harnesses before approving a new reporting series. Treat architecture, interface compatibility and measurement equivalence as separate review questions.

The decisive artifact is a migration ledger tied to raw run evidence. It records what was preserved, which mappings were validated, where comparison remains conditional and what changed enough to require a new epoch. Reasoning-token handling, ITL populations, stream buffering, achieved output lengths and duration denominators deserve explicit review because each can change the interpretation of a reported improvement.

Acceptance should reflect the service being purchased or operated. A fixed-length regression profile protects continuity; an arrival-driven profile tests service behavior; a timestamped replay checks a captured workload. Each needs evidence that the client actually delivered its load, together with correctness, latency, errors, throughput and joint SLO attainment.

The worked examples in this report are arithmetic demonstrations, not deployment results. Teams should replace their assumptions with retained observations and uncertainty estimates before accepting a server, runtime or GPU. Keep the old baseline when its contract remains useful, rerun both tools when a bridge is possible, and declare a new epoch when the measurement contract materially changes. That sequence makes migration reviewable and preserves the meaning of historical capacity decisions.