Back to Articles|Published on November 14, 2026|29 min read

GPUSmith Article

Diffusion Model GPU Sizing: Accepted Images and Video by Deadline

Diffusion model GPU sizing for 2026: size private image and video generation by accepted outputs per whole replica and deadline, with SDXL, FLUX, Wan and HunyuanVideo reference points and a worked example.

By GPU Smith

Diffusion Model GPU Sizing: Accepted Images and Video by Deadline

Summary

  1. 01Size diffusion capacity by accepted images or clips per whole replica inside the deadline, including rejected attempts and retries. Public benchmark rates cannot replace a workload-specific acceptance run.
  2. 02Pin the model revision, scheduler, steps, precision, dimensions, seed policy and postprocessing. Changes that affect output must pass quality requalification before their speed gains count.
  3. 03Multi-GPU groups can reduce per-clip latency while consuming more GPU-seconds. In the hypothetical video example, the deadline-only plan favors smaller groups, but a per-clip tail limit can reverse the choice.
  4. 04Measure the complete path from request to stored artifact, including stage memory peaks, failures, retries and matched-window energy. Include cold startup in deadlines that incur it.
  5. 05Compute the replica lower bound from measured accepted rate, add reserve according to the fault policy, and replay a realistic backlog with storage and failure recovery. Publish each configuration with its qualification status.
Inside this article
  1. 01Executive Summary
  2. 02Introduction and Background
  3. 03Key Changes in How Diffusion Capacity Is Sized
  4. 04Implementation Considerations and Process Changes
  5. 05Model and Configuration Reference Points
  6. 06Data Analysis and Evidence
  7. 07Implications and Future Directions
  8. 08Frequently Asked Questions (FAQs)
  9. 09Conclusion

Executive Summary

Sizing a private diffusion platform is a deadline problem measured in accepted deliverables, not a VRAM lookup. The unit that matters is an accepted image or video clip, produced by a whole replica (one execution group of one or more GPUs), after rejections and retries, inside a wall-clock window. This report, current as of November 2026, defines that contract and explains why public benchmark cells cannot replace a measured, workload-specific acceptance run. Every figure below is a dated, attributed third-party number shown with its configuration; none is a new measurement.

The published reference points span orders of magnitude. The MLPerf Inference SDXL benchmark fixes 20 denoising steps and 5,000 validation samples [1], and NVIDIA reported 5 samples/s from eight L40S GPUs [2]. A Stability AI and NVIDIA model card lists H100 SDXL throughput of 0.40 to 0.68 images/s at 30 steps and 1024x1024 [3]. For video, the HunyuanVideo README states a 60 GB minimum for 720px1280px129f [4], Wan2.2 A14B at 1280*720 on one GPU needs at least 80 GB [5], and the Wan2.1 1.3B model needs only 8.19 GB [6]. ML.ENERGY reports 26 kJ to 1.16 MJ per generated video (Source: ml.energy).

Three findings drive the method. First, step count, resolution, precision, scheduler and offloading are part of the product: energy rises linearly with denoising steps [7], so fewer steps are a different quality operating point until requalified. Second, parallelism inside a request consumes GPUs without guaranteeing speed. xDiT states that more ranks do not guarantee lower latency [8], and the HunyuanVideo README's 8-GPU latency of 337.58 s versus 1,904.08 s on one GPU is a 5.64x speedup on 8x the hardware. Third, batching is not free: on a B200 at 28 steps, Flux.1-Dev batch 8 took 15.57 s against 2.10 s at batch 1 [9].

The arithmetic is short. Accepted replica rate is M/T; replicas needed are ceil(required accepted outputs divided by deadline seconds times that rate); procurement is replicas times GPUs per replica plus reserve that follows the stated fault policy. Expected attempts per accepted output are 1/p under independent retries [10]. In the worked example, the same 600-clip, 8-hour requirement needs 50 GPUs as single-GPU replicas but 72 as 8-GPU replicas, and the answer flips if a per-clip tail limit applies. The deliverable described here is a workbook template, with the tabs listed later in this report, that publishes each configuration cell as pass, quality fail, unsupported or evidence missing.

50 GPUsSingle-GPU fleet in the hypothetical 600-clip, 8-hour example, without reserve
72GPUs in the hypothetical 600-clip, 8-hour plan using 8-GPU replicas, without reserve
36.166 GBReported FLUX.1 [dev] BF16 peak memory on an H100
21.5 sReported H100 Flux latency with model CPU offload, versus 6.7 s for the same pipeline

Introduction and Background

Enterprise creative and media automation teams that move image or video generation onto owned GPUs face a question that generic GPU lists do not answer: what is the smallest practical pool that delivers a stated number of accepted outputs by a stated deadline? This report covers diffusion inference only. It excludes diffusion language models, video understanding inputs, training and fine-tuning, and it is not a general best-GPU list. It is written from the perspective of GPU Smith, a private AI infrastructure engineering consultancy. All configuration figures come from third-party sources; none are GPU Smith measurements.

Existing guidance already explains why video needs more memory than images. A GPU-cloud vendor's September 2026 guide gives a fleet formula of clips per hour times per-clip runtime times concurrency per GPU, which reads as vendor shorthand with unstated unit conventions (this report's formula below divides by the accepted replica rate instead) [11] and advises taking runtime and concurrency from a pilot rather than a published table [11]. This report starts from that pilot-first advice and adds the denominator buyers are ultimately held to: outputs that pass review. The same guide states that doubling resolution or frame count roughly doubles the activation-dominated portion while weights stay constant [11]; that is a general heuristic from a single vendor page, and the measured data later in this report show why scaling rules of that kind need validation on the exact runtime path.

The best public anchor for complete-output accounting is the ML.ENERGY benchmark. Its paper reports energy at the granularity of a whole generation response [7], and its January 2026 results post defines a diffusion response as one generated image or video (Source: ml.energy). The v3.0 release covers 46 models across 7 tasks and 1,858 configurations on H100 and B200 GPUs (Source: ml.energy). The paper's data describe early-2025 runs on A100 40 GB and H100 80 GB GPUs [7], so this report treats the paper and the later post as separate vintages and does not merge their numbers. The response-level framing has been reused as a motivating statistic in a USENIX ATC 2025 paper [12], and NVIDIA's June 2026 efficiency guidance points readers to benchmark-driven comparisons [13]. Those are citations of the reference, not validation of any sizing result here.

Key Changes in How Diffusion Capacity Is Sized

From generations to accepted deliverables

The first change is the denominator. A GPU that produces 1,000 images per hour has delivered 1,000 accepted images only if every one passes the buyer's rubric. The ML.ENERGY paper makes a related point for latency: a diffusion result is useful only when the full image exists, so latency constraints apply to whole-image generation [7]. Acceptance extends the logic: a clip that is generated, decoded and encoded but rejected has consumed GPU time and delivered nothing.

Acceptance needs a named owner and rubric. Automated metrics are partial proxies. FID was introduced to capture the similarity of generated images to real ones better than the Inception Score [14], yet a later study reports that FID contradicts human raters on text-to-image models [15]. For video, VBench decomposes quality into 16 dimensions [16], and its authors state that existing metrics do not fully align with human perceptions [16]. The acceptance rate should therefore come from a fixed prompt and seed set scored by the party that will actually accept the media, with automated scores used as screens.

Human raters need a reliability check. Cohen's kappa is documented as a statistic that measures inter-annotator agreement [17], and a peer-reviewed tutorial suggests no fewer than 30 comparisons [18] and points to Krippendorff's alpha for multiple raters [18]. An audit with fewer than 30 doubly rated items would leave the acceptance rate poorly bounded.

From per-GPU numbers to whole replicas

The second change is the unit of capacity. xDiT distinguishes data parallelism, which runs independent prompts on separate model groups, from within-request strategies (Ulysses, Ring, PipeFusion, tensor and classifier-free-guidance parallelism) whose ranks cooperate on one output. Its launcher computes the GPU count of one group as the product of the parallel degrees, doubled when CFG parallelism is set [19]. Diffusers' data-parallel path replicates the same model on each GPU and gives each GPU different prompts [20].

A multi-GPU renderer is therefore one execution group, not several independent per-GPU jobs. Counting it as G units overstates concurrency; counting only its latency gain understates its cost. Layouts are also constrained by the model. xDiT requires the Ulysses degree to divide the query and key-value head counts unless the model supports head padding [8], and Diffusers notes that Ulysses cannot use more GPUs than the model has attention heads [21]. Diffusers says tensor parallelism needs a fast interconnect such as NVLink [22]. The L40S has no NVLink, while the H100 SXM offers 900 GB/s NVLink (per NVIDIA's specification tables [23] [24]), so the same layout can be a pass on one platform and unsupported on another.

From one stage to the full pipeline

The third change is stage accounting. A deliverable passes through prompt encoding, iterative denoising, VAE decoding, optional upscaling or interpolation, encoding, transfer and storage, and optional checks. NVIDIA states that roughly 90% of end-to-end SDXL processing is spent in the iterative UNet denoising steps [2]. That dominance is configuration-specific: in the SDXL TensorRT card's four-step Latent Consistency Model timings, the VAE stage on an H100 (126.95 ms) is longer than the UNet (102.8 ms) [3]. When step count falls, the stage mix changes, and a profile taken at 50 steps does not describe a 4-step product.

Video adds stages with their own resources. FFmpeg documents frame-rate conversion by motion interpolation [25] as a separate filter from duplication or dropping of frames [25], so frames rendered by the model, frames after interpolation and the container frame rate are three different quantities. NVIDIA notes that encoding can use dedicated hardware on the GPU [26], so whether an encode stage competes with the denoiser depends on the runtime path. Moderation is per model: Diffusers describes the safety checker as a post-generation check on the output image [27], so its rejections add retry work after the GPU time is spent.

From defaults to a pinned output contract

The fourth change is that every output-changing option belongs to the contract. Diffusers advises starting from the checkpoint's default scheduler and swapping only for a deliberate speed or quality tradeoff [28]. Step count is a model-variant property: guidance-distilled FLUX.1 takes about 50 sampling steps for good-quality generation [29], while FLUX.1 [schnell] is documented to produce high-quality images in 1 to 4 steps [30]. The ML.ENERGY paper observes that beyond some number of steps the gain may be virtually indistinguishable to human users [7], while energy keeps rising linearly [7]. Fewer steps, lower precision or a different scheduler should be treated as a new configuration that must pass requalification.

Reproducibility rules constrain the seed policy. PyTorch states that completely reproducible results are not guaranteed across releases [31] and may differ between CPU and GPU executions even with identical seeds [31]. Diffusers recommends a CPU generator because CPU random number generation is more stable across machines [32]. A retry policy must decide explicitly whether an attempt replays the same seed or draws a new one, and full determinism has a price, since deterministic operations are often slower [31].

Speed that comes from a lossy setting cannot be banked until the accepted fraction is remeasured.

Implementation Considerations and Process Changes

Figure 01
From output contract to a replayed capacity plan
  1. 01Define the output contract

    Fix the output requirements and acceptance rubric before comparing configurations.

  2. 02Pin and screen configurations

    Fix the runtime and output-changing options, and identify unsupported configurations before timing.

  3. 03Qualify quality

    Use a representative prompt and seed set with a review protocol, varying one factor at a time.

  4. 04Measure accepted replica rate

    Record end-to-end accepted rate per replica, with cold startup included when the deadline incurs it.

  5. 05Compute capacity and reserve

    Derive the replica lower bound and add the reserve required by the fault policy.

  6. 06Replay the backlog

    Validate the plan with output storage, failure recovery, batch fill and mixed image and video work.

Pass: the cell met the acceptance rate and tail target on the pinned contract, with artifacts hashed.

Publish quality fail, unsupported or evidence missing when the corresponding qualification status applies.

Build the output-contract and stage ledger first

Compare configurations only after the contract is written. Table 1 below lists the fields to fix before any timing is recorded.

Ledger fieldWhat to recordWhy it changes capacity
Image dimensionsNative width and height, not post-upscale sizeMLPerf SDXL generates 1024x1024 by default [1]; a new shape triggers recompilation [33]
Video frames and rateRendered frames, interpolation factor, container rateWan's VAE follows a 1 + T frame format [34]; LTX-Video requires 8n + 1 frames [35]
Scheduler and stepsScheduler class, step count, guidance, flow shiftML.ENERGY sweeps steps as a first-class knob [7]
Precision and kernelsbf16, fp8, fp4, attention backendReduced precision can change output quality [36]
Model revision and licenseCheckpoint hash, license termsFLUX.1 [dev] generated outputs may be used commercially, subject to the license's output terms, but the weights fall under the FLUX.1 [dev] Non-Commercial License, so hosting them in a commercial pipeline is governed by separate terms that an enterprise should confirm with the licensor or counsel [37]; HunyuanVideo's license excludes the EU, UK and South Korea [38]
Seed policyGenerator device, seed per attempt, retry ruleResults can differ between CPU and GPU even with identical seeds [31]
PostprocessingUpscale, interpolate, encode settingsSlower x264 presets give better compression [39]
AcceptanceRubric, rater count, decision ownerMetrics are proxies [15]
GPU groupGPUs per replica, parallel degrees, interconnectGroup size is the product of the degrees [19]

Table 1 is a ledger, not a benchmark. Its value is that each row answers a question that is otherwise answered implicitly by whoever ran the test. A result recorded without these fields cannot be reproduced after a driver, runtime or model-revision change, and PyTorch's notes explain why identical seeds will not rescue it [31].

For each pipeline stage, also record which resource it uses and whether it overlaps with another stage:

  • Prompt encoding: text encoder on the rendering GPU group, a separate GPU or CPU.
  • Denoising: the iterative transformer or UNet passes, usually the dominant term.
  • VAE decoding: may peak above the denoiser and may be sliced or tiled.
  • Upscaling or interpolation: separate models or filters with their own memory.
  • Encoding: CPU encoders or GPU hardware encoders, with their own throughput.
  • Transfer and storage: network, object storage and retention.
  • Checks: moderation and quality screens that can trigger retries.

Pin the runtime and enumerate unsupported cells

Pin the model, runtime, container and every output-changing option, then enumerate the cells that are known unsupported before measuring. Diffusers warns that group offloading may fail on models whose forward pass contains weight-dependent device casting of inputs [40]. Memory stages need separate peaks. VAE tiling divides an image into overlapping tiles [40], but decoded tiles may show tone variation [40], and xDiT says tile-local normalization and blending can change the output [41]. Tiling is therefore a quality-affecting option to requalify, not a free memory saving.

Run a quality-preserving experiment matrix

Fix a representative prompt and seed set and a review protocol, then vary one factor at a time: steps, dimensions, batch size, precision, offload and hardware. Each cell ends in one of four statuses.

  • Pass: the cell met the acceptance rate and tail target on the pinned contract, with artifacts hashed.
  • Quality fail: the cell ran but the accepted fraction or rubric score fell below the contract.
  • Unsupported: the runtime refused the layout, such as a head count the Ulysses degree does not divide [8], or a container that fails at startup when the GPU lacks memory for the requested precision [42].
  • Evidence missing: no run exists. The cell stays on the sheet so nobody infers a result from a neighboring cell.

Lower precision deserves its own gate. NVIDIA reports that for FLUX.1 Kontext [dev] the transformer backbone saves approximately 2x memory at FP8 and 3x at FP4 [43], and its Torch-TensorRT results attach a speed gain to the precision change (on a B200 at 20 steps, 6.56 s for a batch of two images at the PyTorch baseline [44] versus 2.72 s with FP8 [44]). Speed that comes from a lossy setting cannot be banked until the accepted fraction is remeasured.

Measure end to end, and keep warmup honest

Record the following for every cell:

  • End-to-end batch time: wall clock from request to stored artifact.
  • GPU memory peak: per stage, not only the whole-pipeline maximum.
  • Host memory and transfer: offload traffic and pinned-memory needs.
  • Accepted outputs: attempted, accepted and rejected counts with reasons.
  • Failures and retries: including out-of-memory events and restarts.
  • Energy: joules over a window matched to the accepted-output count.

Keep cold-model startup separate from steady operation, but include it in any deadline that actually incurs it. Compilation is the largest hidden term: compiling the Flux transformer cut H100 latency from 6.7 s to 4.5 s [45], while regional compilation took 9.6 s cold and 2.4 s with a warm cache [45]. Cache artifacts are valid only for the same PyTorch and Triton versions [46], and xDiT recommends reusing input shapes to lower compilation cost per run [47]. A mixed-shape queue pays that cost repeatedly.

Benchmark runners usually exclude warmup. xDiT performs complete untimed pipeline calls before timing [47], and the Wan2.2 README reports averages of multiple samples taken after the warm-up phase [5]. That is good steady-state practice but not the experience of the first request after a restart.

Size from measured accepted rate, then replay

Size only from measured accepted rate, then run a realistic backlog replay with output storage and failure recovery. Validate batch fill and mixed image and video interference in the replay, because isolated throughputs do not add. NVIDIA's NIM documentation notes that Wan2.2 containers select multi-GPU parallelism by GPU count [42], so a pool that mixes group sizes changes layouts, and therefore times.

Queueing sets headroom. Little's Law applies to the whole queueing system or to the queue or service facility alone [48], and in a single-server queue the time in system grows without bound as utilization approaches one [49]. Retried work shares the pool with fresh work: the Google SRE book warns that retries can amplify low error rates into higher levels of traffic (Source: sre.google) and that queued requests consume memory and increase latency (Source: sre.google). Reserve capacity follows the fault policy; the SRE book describes provisioning to handle a simultaneous planned and unplanned outage (Source: sre.google), which for GPU groups means whole replicas, not individual cards.

Output volume belongs in the replay. YouTube's published SDR upload recommendations give 8 Mbps for 1080p at standard frame rates [50] and are framed as recommendations, not requirements [50]. At 8 Mbps, one second of video is about 1 MB, so a 5-second 1080p clip is about 5 MB (arithmetic from the published bitrate, not a measured file size). Rejected generations, intermediates and logs need a retention policy, and storage throughput should be exercised by the same replay as the GPUs.

Model and Configuration Reference Points

Table 2 below collects dated, attributed reference points with their stated conditions. They are not predictions for any other system: each number applies only to the model, precision, resolution, step count, batch size and hardware named in its source, and several sources do not state the batch size.

Model and sourceStated conditionsMemory statementTiming or throughput statement
SDXL, MLPerf Inference1024x1024, 20 steps, 5,000 validation samples [1]Not stated in the benchmark description8x L40S: 5 samples/s offline [2]; 8x H200 at 700 W: 13.7 samples/s offline [2]
SDXL, TensorRT model card30 steps, 1024x1024; baseline vs TensorRT [3]Not statedA100: 0.27 to 0.36 images/s; H100: 0.40 to 0.68 images/s; A10: 0.10 to 0.12 images/s
FLUX.1 [dev], Hugging Face blogH100 80 GB, 1024x1024, 50 steps, guidance 3.5 [51]BF16 peak 36.166 GB; pipeline weights about 31.447 GB [51]BF16 12 s; 8-bit bitsandbytes 27 s
FLUX.1 [dev], PyTorch blogH100, 28 steps, bf16 [45]About 33 GB; CPU offload peak 22.7 GB6.7 s baseline, 4.5 s compiled, 21.5 s with model CPU offload [45]
FLUX.1 [dev], PyTorch Blackwell blogB200, 1024x1024, 28 steps [9]BF16 38.34 GB at batch 1; 53.00 GB at batch 82.10 s at batch 1; 15.57 s at batch 8
Wan2.1 T2V-1.3B, vendor README480P, RTX 4090 [6]8.19 GB [6]About 4 minutes for a 5-second video
Wan2.2 TI2V-5B and A14B, vendor README1280704 (5B) with offload flags [5]; 1280720 (A14B), single GPU [5]24 GB minimum (5B); 80 GB minimum (A14B)5B: under 9 minutes for a 5-second 720P video [5]
HunyuanVideo, vendor READMEBatch size 1, 129 frames, 50 steps [4]60 GB at 720px1280px129f; 45 GB at 544px960px129f; FP8 weights save about 10 GB [4]1,904.08 s on 1 GPU; 337.58 s on 8 GPUs (GPU type not stated in the README text) [4]
LTX-2, PyTorch Blackwell blogB200, 768x512, 121 frames, 24 fps, 40 steps, VAE tiling [9]BF16 72.77 GB at batch 1; 107.40 GB at batch 816.230 s at batch 1; 122.427 s at batch 8 [9]

The table shows how little transfers between rows. A FLUX.1 [dev] BF16 pipeline of about 31 GB of weights peaks at 36 GB on an H100 at batch 1 [51], and the Diffusers documentation says Flux needs about 50 GB of RAM or VRAM to load all components unoptimized [29]. Different peak definitions (weights, peak activation, whole-pipeline load) yield different "requirements" for the same model. The two H100 Flux entries agree on per-step cost: 6.7 s over 28 steps and 12 s over 50 steps are both about 0.24 s per step, which supports a linear step term for the denoiser on that setup but says nothing about other hardware.

Images: what the numbers support

For images per hour per GPU, the published figures are system-level or configuration-specific. The MLPerf results are for eight-GPU systems, so dividing by eight gives about 0.63 samples/s per L40S (2,250 per hour) and about 1.71 samples/s per H200 (6,165 per hour); these are derived values that assume even scaling inside the system, not published per-GPU results. MLPerf SDXL also binds quality: a valid submission must achieve FID and CLIP scores within a stated range [1], and MLCommons added a human compliance check [1], which is the accepted-output principle at benchmark scale. The model card figures translate to 1,440 to 2,448 images per hour per H100 at 30 steps (0.40 to 0.68 images/s times 3,600), but the card states neither batch size nor acceptance criteria.

For H100 versus L40S, the published sources cited here do not report both on the same model and settings. MLPerf v4.0's nearest pair is an 8x H200 system against an 8x L40S system, 13.7 versus 5 samples/s (about 2.7x) on SDXL at 20 steps; the H200 shares the Hopper architecture with the H100 but carries more memory. The H100 SXM lists 80 GB and 3.35 TB/s of memory bandwidth against 48 GB of GDDR6 and 864 GB/s for the L40S [23] [24]. NVIDIA's NIM support matrix lists FLUX.1-dev engines for both an H100 SXM and an L40S [42]. The ratio is a hypothesis to test per workload, not a constant.

For Flux class models, a single 1024x1024 image usually needs one GPU: FLUX.1 [dev] is a 12 billion parameter rectified flow transformer [37] and fits in about 36 GB at BF16 on an H100 (Table 2). FLUX.2 [dev] is a 32 billion parameter model [52] paired with a 24B parameter vision-language text encoder [53], so its memory and layout need fresh qualification rather than extrapolation from FLUX.1.

Video: why memory and time do not scale from one number

Wan2.1's 1.3B model is recommended at 480P although it can produce 720P [6]; Wan2.2's A14B models need at least 80 GB for a single-GPU 720P run, and the 5B model runs on 24 GB with offload flags (Table 2). The Wan paper states that frame counts follow a 1 + T format [34], that typical sequences are 81 frames for 1280 x 720 video [34], and that the VAE limits frames per processing chunk to at most 4 to avoid memory overflow [34]. Part of the decode memory is therefore bounded by design, one reason a linear memory model from frame count is unreliable.

Measured batch data confirm that scaling is not proportional. For LTX-2 on a B200, batch 8 took 122.427 s against 16.230 s at batch 1, or 15.3 s per video, only about 6% better per clip, while peak memory rose from 72.77 GB to 107.40 GB (about 48%) [9]. For Flux.1-Dev on the same GPU, batch 8 gave 1.95 s per image against 2.10 s, about 7% better, with 38% more peak memory. Batching bought little throughput at a large memory cost, consistent with the PyTorch team's description of diffusion models as heavily compute-bound [54] and ML.ENERGY's observation that diffusion models cannot run as many concurrent generations as language models [7].

Video also has contract traps. LTX-Video works best on resolutions under 720 x 1280 and below 257 frames [35], and Wan2.1's 14B text-to-video task samples 50 steps while image-to-video uses 40 [6], so a mixed queue has different per-job costs even at one resolution. A rack-scale anchor exists but does not translate to a small pool: NVIDIA's September 2026 blog on MLPerf Inference v6.1 reports 0.65 720p videos per second at 5.7 seconds per video on a GB300 NVL72 system [55], with the benchmark using Wan-2.2-T2V-A14B and the VBench dataset [56]. The source does not state whether this result is for the server or offline scenario, so no per-GPU rate is inferred.

Data Analysis and Evidence

The evidence falls into three layers: benchmark methodology (what a number counts), configuration-specific timings (what it was measured on) and the arithmetic that converts a measured rate into hardware. The first two are third-party; the third is this report's contribution and is labeled as such.

What the leaderboard layer does and does not show

ML.ENERGY's v2 paper measures time and energy per configuration with the Zeus library [7], and because diffusion models are batched as a whole, per-request energy is the whole-batch energy divided by the batch size [7]. The January 2026 post used default denoising steps, resolution and frames for all models (Source: ml.energy), reports text-to-image models from 0.6B to 12B parameters with 20 to 50 denoising steps (Source: ml.energy), and reports 26 kJ to 1.16 MJ for a single video (Source: ml.energy). For B200 versus H100 at matched latency, B200 won 86% (18 of 21) of text-to-image comparisons with a median 15% energy reduction (Source: ml.energy), using whole-generation latency constraints of 10, 30 and 60 s for text-to-image and 100, 500 and 1,000 s for text-to-video (Source: ml.energy), a useful template for stating deadlines per output rather than per token.

The limits matter as much. The leaderboard is interactive, so this report cites no current slider values or computed winners, and the paper reports energy mostly in figures. Neither source reports rejection or retry rates. They support the framing (complete outputs, step count as a knob, batch-level measurement), not a fleet size.

The planning calculation

For M accepted generations measured in T seconds on a G-GPU replica:

  • Accepted replica rate: M/T.
  • GPU-seconds per accepted output: G times T divided by M, with idle and cold-start scope declared.
  • Replica lower bound: ceil(required accepted outputs divided by (deadline seconds times the observed accepted replica rate)).
  • Procurement count: replicas times G plus an explicitly justified reserve.
  • Energy per accepted output: measured matched-window joules divided by accepted outputs; kWh is joules divided by 3,600,000.

If a rejection model is used for planning, expected attempts per accepted output are 1/p, because the attempt number of the first success in Bernoulli trials is geometric [10]. That assumes independent attempts and a stationary acceptance rate, which fail when a bad prompt batch or a changed model revision shifts p; replace the model with measured accepted throughput as soon as it exists. For heterogeneous work, sum empirical replica-seconds per cell and confirm with a replay.

Image example (Hypothetical Example)

Assume a 1,024x1,024 contract with a fixed revision, scheduler and step count. A single-GPU replica is measured for 3,600 s: 2,400 attempted images, 2,040 accepted (85%). The accepted rate is 2,040/3,600 = 0.567 per second. The requirement is 20,000 accepted images in 8 hours (28,800 s). One replica delivers 16,320 accepted images in that window, so replicas = ceil(20,000/16,320) = 2, running at about 61% of capacity. With an N+1 reserve, procurement is 3 GPUs. If the matched window used 1,020,000 J, energy per accepted image is 500 J and the job totals 10,000,000 J, about 2.78 kWh. Every number here is invented for illustration and none is a benchmark.

Video example (Hypothetical Example)

This example combines the latencies published in the HunyuanVideo README (1280x720, 129 frames, 50 steps, 1 to 8 GPUs via xDiT) [4] with an assumed 80% acceptance rate, a 600-clip requirement and an 8-hour (28,800 s) deadline. The README does not state the GPU type, its table caption reads "on 8xGPU" while its only hardware statement is that the model "is tested on a single 80G GPU", so the test setup is ambiguous and the 1-GPU row inherits that uncertainty; Table 3 is purely illustrative. The 3, 5 and 6 GPU layouts it also lists are omitted. Table 3 below shows the result.

LayoutLatency per attemptSpeedupGPU-seconds per attemptAccepted per replica in 8 hReplicas for 600GPUs (no reserve)
1 GPU per replica1,904.08 s1.00x1,90412.105050
2 GPUs per replica934.09 s2.04x1,86824.662550
4 GPUs per replica514.08 s3.70x2,05644.821456
8 GPUs per replica337.58 s5.64x2,70168.25972

Table 3 shows that if the objective is only 600 accepted clips in 8 hours, the 1-GPU and 2-GPU layouts tie at 50 GPUs and the 8-GPU layout needs 72, about 44% more hardware, because the 8-GPU group spends about 42% more GPU-seconds per clip. The result reverses under a per-clip latency limit: a 1-GPU attempt takes about 32 minutes and an 8-GPU attempt about 5.6 minutes, so a 15-minute tail target rules out the single-GPU layout. Reserve granularity differs too: one spare replica adds 1 GPU to the 1-GPU plan (51) but 8 GPUs to the 8-GPU plan (80). The 1-GPU row also requires that the model fit on one GPU, which the README's 60 GB minimum for this shape makes plausible on an 80 GB card. Because acceptance and speedup interact with the real runtime path, this table is a template, not a recommendation.

Cross-checking published speedups

Published multi-GPU gains support caution. In Diffusers' context-parallel benchmark on 4 H100 GPUs at 50 steps per image, Ulysses took 6,670.789 ms per image while Ring took 13,076.492 ms [21], so the strategy choice on the same four GPUs changed time by almost 2x. xDiT says each additional device adds communication and synchronization, so the workload should be measured [57], and a hybrid layout such as two CFG branches each with four Ulysses ranks uses eight devices for one request [57]. Its determinism check compares timed outputs against the first run with exact equality [47], a useful regression test when changing layouts.

Figure 02
Hypothetical GPUs for 600 clips in 8 hours at 80% acceptance, without reserveGPUs (no reserve)
Source: Table 3

Implications and Future Directions

For procurement teams, the number to request from a vendor or produce in a pilot is accepted outputs per whole replica per deadline window, with the contract attached, not raw frames or generic throughput. Single-GPU replicas favor throughput deadlines when the model fits, multi-GPU groups buy tail latency at a GPU-seconds premium, and reserve granularity grows with group size. Offloading can turn a fit into a slowdown: PyTorch's H100 measurements show model CPU offload taking 21.5 s against 6.7 s for the same Flux pipeline while peak memory fell from about 33 GB to 22.7 GB [45].

For platform engineers, stage ownership should be explicit. Whether encoding, interpolation and moderation run on the rendering group, on separate GPUs or on CPUs determines whether they overlap with denoising or add to it, and overlapping stages cannot simply be summed. Warmup and compile policy deserve a service-level decision, since a resolution change triggers recompilation [33] and cache artifacts are version-bound [46].

Three uncertainties remain. Benchmark coverage is moving: MLPerf Inference now lists a Wan 2.2 text-to-video workload [56], but results must be read for their stated scenario. Model families change quickly, and FLUX.2 [dev] at 32 billion parameters [52] shows a successor can change memory needs several-fold, so each sizing sheet should carry a revision stamp. And the quality measures in common use remain proxies [16], so the acceptance owner's rubric is the durable part of the workbook. Keep the workbook under version control so a runtime, driver or hardware change triggers a re-run of the affected cells rather than a re-argument. A minimal workbook has these tabs:

  • Output contract: the ledger in Table 1, with model revision and rubric.
  • Configuration matrix: model, shape, scheduler, steps, precision, GPU group and runtime per cell.
  • Stage timings: end-to-end and per-stage times, memory peaks and energy window.
  • Attempt log: attempted, accepted, rejected and retried counts with decisions and reviewers.
  • Sensitivity: deadline, reserve and acceptance-rate variations, as in Table 3.
  • Provenance: artifact hashes, timestamps and re-run instructions.

Frequently Asked Questions (FAQs)

How many GPUs do I need for Stable Diffusion at volume?

Divide the accepted outputs required by the deadline in seconds times the measured accepted replica rate, round up, and multiply by GPUs per replica. For a single-GPU SDXL or Flux replica the replica count equals the GPU count. The published SDXL range of 0.40 to 0.68 images/s on an H100 at 30 steps [3] is a starting hypothesis only, because it carries no acceptance rate.

What are the self-hosted image generation GPU requirements, and the FLUX VRAM figures?

They depend on the model, precision and options. FLUX.1 [dev] at BF16 measured a 36.166 GB peak on an H100 [51]. Quantization and offload lower memory but change speed and sometimes output, so each is a separate workbook cell.

What GPU memory does Wan 2.1 video generation need?

The vendor README states 8.19 GB for the 1.3B model [6]. Larger Wan models need far more, and HunyuanVideo states 60 GB for 720px1280px129f, so treat any single figure as valid only for its stated shape.

How do I size GPUs for private image generation and plan on-premise capacity?

Write the output contract, measure accepted outputs per replica on the pinned pipeline, add a measured retry term, set reserve by fault policy, then replay a realistic backlog with storage and failure recovery.

H100 or L40S for image generation throughput?

No same-settings public pair is cited here. MLPerf v4.0 compares eight L40S GPUs (5 samples/s) with eight H200 GPUs (13.7 samples/s) on SDXL (Table 2). Test both on your contract.

Conclusion

Diffusion GPU sizing is a measured-throughput exercise over a defined output contract. The unit is the accepted deliverable per whole replica per deadline window, after rejection and retries, and the replica is the full execution group that renders it. The public record supplies methodology and reference points, such as complete-output accounting in ML.ENERGY, fixed-step MLPerf SDXL rules and vendor memory statements, but each is tied to a model, shape, precision and hardware that may not match a production mix.

The practical path is short:

  • Define: write the output contract and acceptance owner.
  • Pin: fix the runtime and every output-changing option.
  • Qualify: run a quality-preserving matrix with a rater-checked acceptance rate.
  • Measure: record end-to-end accepted rate per replica.
  • Compute: derive the replica lower bound and the reserve the fault policy requires.
  • Replay: run a backlog with storage and failures.
  • Publish: mark each cell pass, quality fail, unsupported or evidence missing.

A sheet built that way survives hardware and runtime changes because every number carries its conditions.

External Sources (57)

About

GPUSmith

Plan and build private AI compute with GPU Smith. We connect workload requirements with GPU hardware, networking, deployment and operating decisions so your infrastructure fits the work it must perform.

GPU Smith provides independent engineering and research for private AI infrastructure. We help technical buyers and operators reason about GPU workload sizing, hardware selection, procurement, deployment and inference operations, from the compute system to the networking, power and cooling requirements around it.

Start with the workload

Useful infrastructure decisions begin with the models, data, throughput, latency and operating constraints a team actually has. GPU Smith helps connect those requirements with system design and deployment choices. The goal is a privately controlled AI environment whose capacity and operational demands are understood before a purchasing decision.

Hardware and supplier research

Our public reference library covers GPUs, complete systems and networking components. The server-vendor directory and manufacturing research help teams investigate suppliers, compare options and follow sources behind technical claims. We distinguish manufacturer specifications from measured benchmarks and advertised capabilities from tested configurations. Reference pages are research resources, not a live inventory listing or a binding equipment quote.

Deployment and operations

GPU Smith's engineering scope includes infrastructure integration, networking, power, cooling and the practical operation of inference workloads. Our published articles explain the tradeoffs behind deployment, procurement and ongoing operations so teams can ask better questions and document decisions.

Work with GPU Smith

Explore the hardware library, GPU references, networking references and vendor research. Contact GPU Smith with your workloads and deployment constraints to discuss a project and confirm current scope, pricing and availability.

Inclusion of a manufacturer or product in our research does not imply a partnership, certification or endorsement.

Disclaimer

This document is provided for informational purposes only. No representations or warranties are made regarding the accuracy, completeness, or reliability of its contents. Any use of this information is at your own risk. GPUSmith shall not be liable for any damages arising from the use of this document. This content was generated with assistance from artificial intelligence tools, which may contain errors or inaccuracies. Readers should verify critical information independently. All product names, trademarks, and registered trademarks mentioned are property of their respective owners and are used for identification purposes only. Use of these names does not imply endorsement. This document does not constitute professional or legal advice. For specific guidance related to your needs, please consult qualified professionals.