MLPerf Training 6.0 GPU Cost: Time to Quality Worksheet

Executive Summary

MLPerf Training 6.0 GPU cost cannot be read directly from a leaderboard. The benchmark measures wall-clock time to a specified dataset and quality target. ^[1] A buyer must combine a comparable result with a dated system or cloud quote, the full allocation window, and operating assumptions. MLCommons released the cohort on June 16, 2026. ^[2] The additions included DeepSeek V3 and GPT-OSS 20B, both sparse-computation workloads. ^[2] Their inclusion expands workload coverage, rather than establishing a universal ranking for enterprise training.

A public submission supplies a useful, deliberately narrow example. A pinned Dell artifact describes a node containing eight AMD Instinct MI355X accelerators, each with 288 gigabytes (GB) of memory. ^[3] One associated low-rank adaptation (LoRA) fine-tuning log records 487.78 seconds, or 8.13 minutes, and a final evaluation loss below its 0.925 target. Multiplying the accelerator count by that interval gives 1.084 GPU-hours, rounded. ^[4] This is an analyst calculation for one recorded benchmark interval, excluding initialization, other runs, and allocation overhead. It is neither the official aggregated score nor a reproduced measurement from this report.

Public prices illustrate why the commercial boundary matters. The Amazon Web Services (AWS) Capacity Blocks table accessed for this report lists p5.48xlarge at $47.757 per instance-hour. ^[5] CoreWeave lists a North America eight-GPU HGX H100 node at $49.24 per hour on demand. ^[6] Those are different purchasing models. Neither rate prices the Dell run, which uses different accelerators and a different system. AWS also separates reservation and operating-system fees. ^[5] A credible cost comparison therefore needs matched configurations and billing terms before multiplying time by price.

The procurement recommendation is to shortlist on relevant, valid time to quality, then authorize purchase only after an auditable pilot. Preserve the submission revision, system description, configuration, convergence logs, repeated-run spread, commercial quote, and measured energy boundary. Throughput charts remain useful screening evidence: Lambda explicitly measures samples processed per second, whereas the training benchmark measures reaching quality. GPU telemetry also has a narrower boundary than whole-system or facility energy. Unknown acquisition prices, staffing effort, and facility consumption should remain explicit worksheet blanks until measured or quoted. ^{[7] [8]}

Introduction and Background

An enterprise buyer of graphics processing units (GPUs) needs an acceptable trained model delivered within a practical budget and schedule. A faster accelerator can be relevant to that decision, but the procurement unit is usually a usable system: processors, memory, networking, storage, software, and access to capacity. The central question is which public result provides enough evidence to justify a paid pilot, followed by which pilot observations justify purchase.

MLPerf Training supplies a defined endpoint: training a model on a specified dataset until the required quality is achieved. ^{[1] [9]} That endpoint distinguishes useful model progress from processing examples quickly. It does not, by itself, provide a commercial quote or show how a buyer's private training workload will converge.

This report treats GPU Smith as an adjacent independent engineering adviser. Its perspective is an evidence standard for procurement, rather than an accelerator or cloud option in the comparison. The analysis covers model training, including a public fine-tuning example. It does not use inference serving performance to estimate training time.

The analysis is anchored to October 21, 2026 access checks. Version control matters because the current benchmark landing page already includes workloads marked v6.1, alongside retained v6.0 entries. The worked example therefore uses immutable submission links rather than treating the entire current page as the historical cohort. ^[1]

Readers should distinguish three kinds of statement throughout: a benchmark body's definition, a submitter's recorded result, and an analyst's procurement calculation. The selected submission was inspected, not executed again. No facility meter readings or buyer acquisition quotes were obtained. Those omissions limit the report's cost conclusion: it can demonstrate the worksheet and identify evidence requirements, but it cannot declare a purchased system's measured cost to train.

Key Changes

Sparse workloads make task selection more consequential

The release added DeepSeek V3 and GPT-OSS 20B, extending coverage of mixture-of-experts (MoE) training. MoE models route tokens through selected experts, so a procurement test should examine the relevant sparse workload rather than assume a dense-model result transfers automatically. The release also describes the smaller new benchmark as a representative portion of end-to-end training. Its elapsed time should not be advertised as the cost or duration of complete production pretraining. ^[2]

The practical change is a broader choice of evidence. A buyer running sparse pretraining should seek a sparse submission with relevant scale and topology. A buyer adapting an existing model should begin with a fine-tuning task and a suitable quality threshold. Matching the task is the first filter; accelerator branding comes later.

The result is a system and software observation

Closed division requires the reference preprocessing, model, training method, and quality target. Open division permits alternative data, preprocessing, models, or methods. Both can be informative, but their claims need different interpretation. The current official rules provide these definitions; buyers should retain the rules applicable to the original round rather than assume the latest document freezes every historical detail. ^[10]

This distinction prevents an overly broad claim that Closed makes every implementation identical. It establishes controlled problem definitions while still leaving system and implementation details important to a procurement decision. Open evidence can support an interesting alternative method, provided the report explains the changed problem or implementation.

Availability and validity belong in the shortlist

MLCommons distinguishes systems containing components available for purchase or cloud rental from preview configurations. Availability classification helps screen what can be ordered, but it does not replace a supplier's confirmation of delivery, capacity, or the exact configuration. ^[1]

Published results also require a status check. MLCommons maintains a change log for modifications after publication. The buyer should record when that check occurred and preserve the result used in the decision, rather than rely indefinitely on a launch announcement. ^[11]

A useful intake checklist is:

• Task: Specify pretraining, fine-tuning, or the exact training operation.
• Endpoint: State the dataset, evaluation procedure, and acceptable quality.
• Division: Preserve the declared comparison category.
• Availability: Verify the configuration can actually be supplied.
• Revision: Retain the submission and applicable rule revisions.
• Status: Check the result change log before issuing the request for proposal.

These are proposed procurement controls. Their purpose is to make the evidence package reconstructable when a reviewer asks why a particular result entered the shortlist.

Implementation Considerations and Process Changes

Separate throughput from time to quality

Throughput answers how many samples a system processes per second. Lambda explicitly includes tokens and images in that definition and notes that implementation affects throughput. Time to quality answers how long training takes to reach a specified evaluation target. Converting between them requires knowing the amount of training needed to converge, not merely the speed of an iteration. ^{[7] [1]}

For initial screening, a throughput chart can identify plausible candidates. Before calculating ratios, however, normalize the model, batch size, numerical precision, optimizer, software, and communication arrangement. AIME explicitly notes that batch size affects both performance and training results. (Source: www.aime.info) Its page also discloses distinct software environments, including PyTorch 2.5.1, CUDA 12.1.105, and Python 3.10.12. (Source: www.aime.info) These details are comparison inputs, not footnotes to discard.

Lambda's page discloses a legacy environment containing Ubuntu 20.04, PyTorch 1.13.0a0+d0d6b1f, CUDA 11.8.0, and cuDNN 8.6.0.163. ^[7] That disclosure makes the chart interpretable, but it does not turn its values into measurements of a newer submission stack. Avoid creating a GPU speedup ratio from charts that changed software and hardware together.

Read the workload label before using a chart

DigitalOcean's training guide points readers to MLPerf as evidence about training speed to target quality. A LessWrong discussion comment also links Lambda's standardized benchmarks in compute-estimation methodology. These references show how benchmark methods can be reused; they do not independently validate a particular GPU ranking. ^{[9] [12]}

A fetched Runpod comparison page labels its language-model measurements as September 2026 vLLM benchmarks. That is serving evidence rather than training convergence evidence. Procurement teams should classify the measured operation before using any attractive chart in a training business case. ^[13]

The operational comparison checklist is:

• Model: Match architecture and model size.
• Data: Match preprocessing and sequence or sample shape.
• Batch: Record local batch, global batch, and accumulation.
• Precision: Preserve numerical formats and scaling recipes.
• Optimizer: Preserve settings that affect convergence.
• Software: Record framework, libraries, compiler, and container.
• Network: Record fabric, interface selection, and parallelism.
• Metric: Keep throughput and time-to-quality ratios distinct.

Do not combine a training numerator with an inference denominator. A chart may still guide a separate capacity question, but a training cost worksheet should contain evidence for the training operation being purchased.

Comparable Cohorts and the Procurement Evidence Ledger

Decide what comparison the evidence permits

A comparable cohort is a buyer-defined set of results sufficiently similar to support a particular decision. It should be narrower than the full release. Closed division is a useful condition, but procurement also needs workload relevance, system availability, and software that the buyer can obtain. The official division definitions and availability categories address different parts of this problem.

Table 1 below is a proposed comparable-cohort gate. It identifies what to retain before interpreting a ranking.

The table is an evidence filter, not a scorecard assigning universal weights. A mismatch should trigger a documented explanation or a pilot designed to resolve it. The outcome can be a conditional shortlist rather than a forced ranking of systems that solved different problems.

Preserve enough detail to reproduce the decision

The ledger should separate public evidence from buyer observations. The public record may establish a submitted configuration and a successful benchmark interval. The pilot must establish whether the quoted configuration, buyer data path, and operating procedures deliver acceptable results. Keep those two records linked without merging their provenance.

Recommended ledger fields are:

• Identity: URL, repository revision, path, and retrieval date.
• Integrity: Hashes for downloaded code, data manifests, and containers.
• Configuration: Launch arguments, environment, and power settings.
• Convergence: Evaluation sequence, target crossing, and run completion.
• Distribution: All required runs and the scoring calculation.
• Economics: Quote identifier, billing window, storage, and labour.
• Energy: Meter location, units, sampling interval, and timestamps.

These fields are this report's proposed procurement method. They should not be read as a claim that every public submission already supplies buyer-generated hashes, facility measurements, or prices.

Reproducing a Public Submission

What the inspected artifacts actually establish

The worked example uses Dell's versioned 1xXE9785L_8xMI355X system record and its llama2_70b_lora/result_6.txt log. The system record identifies one node with eight MI355X accelerators and classifies it as Closed and Available on-premise. It records Ubuntu 24.04 LTS and ROCm 7.0.1. These are attributes of the submitted system, not a guarantee about every system carrying the same accelerator name. ^[3]

Table 2 below records the evidence boundary for the selected run.

This example demonstrates traceability rather than a vendor recommendation. The retrieved official rules describe discarding the fastest and slowest runs and averaging the remainder. The selected individual log cannot substitute for the cohort's aggregate result, and no aggregate for this selected submission was extracted here. ^[10]

Reproduction procedure for a paid pilot

AMD's vendor-specific reproduction guide covers environment setup, data preparation, configuration, execution, and validation. ^[15] Its LoRA preparation notes 270 GB of disk space for model download and conversion. ^[15] It also documents setting NEXP to 10 for a multi-run result. ^[15] Those details illustrate why preparation and repetition belong in the pilot budget. The guide does not establish transferability to another vendor or configuration.

Start from the exact submission, then apply a controlled procedure:

• Freeze artifacts: Resolve revisions and container digests before scheduling capacity.
• Prepare inputs: Record dataset manifests, checkpoint provenance, and local hashes.
• Verify hardware: Compare the delivered node and fabric with the evidence ledger.
• Execute unchanged: Attempt the submitted configuration before modifying it.
• Retain every run: Preserve successes, incomplete runs, and operational delays.
• Recompute scoring: Apply the correct historical run count and aggregation rule.
• Repeat buyer workload: Evaluate the model task that will authorize purchase.

PyTorch warns that complete reproducibility is not guaranteed across releases, commits, or platforms. It also notes that deterministic operations can be slower. Accordingly, a pilot should report distribution and environment, and should not silently change determinism settings to improve either repeatability or speed. ^[16]

For pilot analysis, report every elapsed time and quality outcome, then provide the median, minimum, maximum, and sample standard deviation alongside the required benchmark aggregate. Keep the complete allocation history even when the official scoring procedure excludes an extreme run. This is a proposed operational report, rather than an additional official scoring requirement.

GPU-Hour Cost and Facility Energy Worksheet

Choose the billed unit before multiplying

For a fixed allocation, GPU-hours = allocated accelerators × allocation hours. For changing allocations, sum that product over each interval. Benchmark GPU-hours use the benchmark interval; billed GPU-hours use the reservation or allocation interval. The public run yields the benchmark-interval quantity calculated below, rather than a complete pilot bill.

Use node-hours × quoted node-hour rate when the supplier sells nodes. Do not add a separate GPU charge when the machine rate already includes it: Google's accelerator-optimized machine pricing explicitly includes GPU cost. Conversely, Google's reservation documentation says reserved resources can be billed even when not started. Billing and useful training time can therefore diverge. ^{[17] [18]}

Table 3 below is the MLPerf Training 6.0 procurement worksheet. Here G is accelerator count, N is node count, T_bench is benchmark seconds, H_alloc is allocated hours, and R_node is currency per node-hour. Information technology (IT) energy uses the declared equipment boundary. All unobserved variables remain inputs to obtain, rather than estimated benchmark facts.

This worksheet supports a cost-to-quality decision only when the corresponding pilot reaches quality. A low hourly rate is insufficient if the allocation takes longer, requires more repetitions, or includes idle reservations. Conversely, faster time may justify a premium when a training deadline has business value. That value should be stated separately from infrastructure cost.

Measure energy at the intended boundary

Power usage effectiveness (PUE) is the ratio of total annual facility energy to annual information technology (IT) energy. Applying a facility overhead factor to a short training run is an allocation assumption, not a direct facility measurement. If facility energy is measured already, do not multiply it by PUE again. ^[21]

Device telemetry can help diagnose utilization and integrate GPU energy. NVIDIA's Data Center GPU Manager (DCGM) aggregates workload statistics across participating GPUs and requires recording before the workload starts. Its GPU boundary does not establish total server, shared storage, network, or cooling energy. ^{[8] [20]}

Data Analysis and Evidence

Public prices provide inputs, not a cost ranking

On the access date, the AWS Capacity Blocks table listed eight-H100 p5.48xlarge at $47.757 per instance-hour, including a US East Atlanta Local Zone entry. Its displayed accelerator-hour equivalent was $5.970, rounded. This is reservation pricing, with a separate operating-system fee, rather than an interchangeable on-demand training quote. ^[5]

CoreWeave's North America page listed eight-GPU HGX H100 at $49.24 per hour and HGX H200 at $50.44 per hour, both on demand. It also states that its individual-GPU pricing applies exclusively to inference-platform customers. A training buyer should therefore use the applicable node price and obtain a dated quote for the required capacity. ^[6]

These are price-only observations. No matched time-to-quality result is paired with either price in this report. They cannot establish which provider delivers cheaper training, and neither may be multiplied by the Dell benchmark's duration to claim a measured cost.

Other opened sources reveal useful missing inputs. Oracle lists an eight-MI355X shape with 288 GB per accelerator, but an attributable hourly price was not recovered. A matching accelerator label would still leave topology and software to verify. Google documents region and zone restrictions; Azure separates managed storage from its VM price. These facts help shape quote requests without filling unknown cells with third-party estimates. ^{[23] [17] [19]}

Quantify what is known and leave the rest conditional

For the selected public log, the calculation is 8 × 487.78 / 3,600 = 1.0839556 GPU-hours before rounding. ^{[3] [4]} If a matching deployment had a genuinely applicable quoted accelerator-hour rate r, its benchmark-interval charge would be 1.0839556 × r. No value for r has been established for that submitted configuration. A node-based quote instead needs the corresponding node duration and actual billing window.

Historical electricity averages illustrate a different uncertainty. The U.S. Energy Information Administration (EIA) reports 2025 industrial and commercial averages of 8.62 and 13.41 cents per kWh, respectively, in its page citing preliminary data from February 2026. ^[22] They are contextual statistics, not the buyer's tariff. Locality and supply conditions affect electricity prices, so the worksheet should use the applicable facility schedule and contractual charges. ^[22]

Power configuration also needs verification. Dell documents an optional permanent cap of 1 kW per GPU for specified XE9785 configurations. That is a configuration capability for the documented system, not a measured draw or proof that the selected XE9785L submission used that cap. Ask explicitly for the delivered system's power settings. ^[24]

The sensitivity report should vary allocation duration, quoted rate, productive utilization, and measured input power independently. Establish low, base, and high inputs from pilot observations and dated quote alternatives. For fixed accelerator count G and time bounds T_low and T_high in hours, the benchmark range is G × T_low through G × T_high GPU-hours. Report the source of each bound; leave the range unpopulated when evidence is missing. Do not describe an assumed range as a statistical confidence interval.

Implications and Future Directions

The main procurement change is procedural: treat a benchmark result as an auditable experimental record, then connect it to a commercial configuration and the buyer's workload. The evidence in Table 2 makes a narrow accelerator-time calculation possible; it leaves the purchase quote and operational evidence to be obtained.

A network-sensitive deployment deserves a network test before a paid training campaign. NVIDIA documents that its Collective Communications Library (NCCL) automatically detects interfaces for inter-node communication. The pilot should preserve interface selection and communication settings alongside the topology, rather than assume a successful container launch verifies the fabric. This advice applies to the relevant software stack, not automatically to every vendor's communication libraries. ^[25]

Checkpoint behavior also matters operationally. PyTorch's distributed checkpoint documentation describes resharding across different cluster topologies. That capability can support recovery planning, but the buyer must test the actual implementation and record the time and storage it consumes. It should not be treated as evidence that the selected public benchmark measured production recovery. ^[26]

For an independent adviser such as GPU Smith, the appropriate purchase gate is a documented acceptance decision. The recommendation is to require:

• Relevance: A declared buyer workload and meaningful quality endpoint.
• Traceability: Public submission artifacts linked to the tested configuration.
• Repeatability: An explained run distribution and preserved convergence evidence.
• Economics: A dated quote joined to measured allocation and operating inputs.
• Operations: Demonstrated data staging, checkpointing, and recovery procedures.
• Facilities: Infrastructure and facilities engineering approval of the measured boundary.

Approval should follow the buyer's predeclared limits for quality, elapsed time, cost, and delivery. If a public result satisfies only the first screening stage, the next authorized expenditure is a bounded pilot. The unresolved evidence should be named in the request for proposal, rather than converted into a generalized promise about accelerator performance.

Frequently Asked Questions (FAQs)

What is MLPerf Training 6.0 time to quality?

It is the elapsed benchmark time needed to reach the specified quality target on the specified dataset. It is not synonymous with a full production training campaign: the release describes a representative training portion for the smaller new sparse workload. Keep the benchmark interval and the buyer's complete job separate. ^{[1] [2]}

How is MLPerf Training 6.0 GPU-hour cost calculated?

Multiply the allocated accelerator count by elapsed hours, then apply a rate only when its system, billing unit, and commercial terms match. Use the observed interval for benchmark accelerator-time, then add separately measured allocation overhead before pricing. The worksheet demonstrates that calculation without assigning an unverified matching rate.

How should an MLPerf Training 6.0 GPU comparison guide procurement?

Use a relevant division, workload, system description, and availability category to form a shortlist. Then reproduce the selected configuration and test the buyer workload under quoted conditions. Open and Closed results can both be useful, but their permitted methodological differences must remain visible. ^[1]

Can a vendor reproduction guide prove another system's cost to train?

A guide provides a procedure for its documented environment. It does not establish another system's convergence, delivered configuration, or commercial price. AMD's reproduction documentation is useful engineering evidence for its own submissions; cross-system conclusions require the corresponding artifacts and pilot observations. ^[15]

Conclusion

MLPerf Training can support procurement when the buyer preserves the experiment it describes. The relevant evidence is a declared workload reaching a declared quality target on a documented system. A throughput chart, an accelerator specification, and an hourly price each answer part of the decision, but they do not automatically combine into a validated training-cost claim.

The public example shows the useful boundary. Its individual duration and accelerator count support a transparent GPU-hour calculation. They do not establish the official aggregate score, the allocation needed for a complete pilot, or the price of a matching deployment. Keeping those distinctions explicit makes the worksheet reusable without implying a purchase recommendation that the evidence does not support.

A disciplined paid pilot closes the remaining gaps. It preserves revisions and hashes, reproduces the submitted configuration, records convergence across runs, measures the appropriate energy boundary, and joins the resulting allocation to a dated quote. It also tests the buyer's data pipeline and operational requirements, because benchmark completion alone does not authorize every production use.

Purchase should follow evidence that satisfies predeclared quality, schedule, cost, and delivery criteria. Until then, unknown prices and operating inputs should remain visible. The strongest shortlist is the one whose unresolved questions can be converted into specific pilot measurements, and whose eventual cost-to-quality conclusion another engineer can reconstruct.