NVIDIA Nemotron 3 Ultra Hardware Requirements: VRAM and Cost
## Executive Summary
NVIDIA's **Nemotron 3 Ultra** (formally **NVIDIA-Nemotron-3-Ultra-550B-A55B**) is a 550-billion-parameter Mixture-of-Experts (MoE) model with 55 billion parameters active per token, released on **June 4, 2026** as the flagship of the Nemotron 3 family <a href="https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/blob/main/README.md#:~:text=550B%20(55B%20active" title="Highlights: 550B (55B active" class="citation-link"><sup>[1]</sup></a>) <a href="https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/#:~:text=550%20billion%20total%20and%2055%20billion%20active%20parameters" title="Highlights: 550 billion total and 55 billion active parameters" class="citation-link"><sup>[2]</sup></a>. It uses a hybrid Mamba-2, Mixture-of-Experts, and Attention architecture NVIDIA calls LatentMoE, supports context windows up to 1,048,576 tokens, and ships in two primary checkpoints: a BF16 (16-bit floating point) release and a quantized NVFP4 (4-bit floating point) release that roughly halves the hardware footprint <a href="https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/#:~:text=Pretrained%20in%20NVFP4" title="Highlights: Pretrained in NVFP4" class="citation-link"><sup>[3]</sup></a>. Hardware requirements differ sharply by precision. NVIDIA's own model card lists a minimum of **8x GB200, B200, GB300, or B300 GPUs, 16x H100, or 8x H200** for the BF16 checkpoint, while the NVFP4 checkpoint drops that floor to **4x GB200, B200, GB300, or B300, or 8x H100** <a href="https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/blob/main/README.md#:~:text=8x%20GB200%2FB200%2FGB300%2FB300%2C%2016x%20H100%2C%208x%20H200" title="Highlights: 8x GB200/B200/GB300/B300, 16x H100, 8x H200" class="citation-link"><sup>[4]</sup></a> <a href="https://build.nvidia.com/nvidia/nemotron-3-ultra-550b-a55b/modelcard#:~:text=4xGB200%2C%204xB200%2C%204x%20GB300%2C%204x%20B300%2C%208xH100" title="Highlights: 4xGB200, 4xB200, 4x GB300, 4x B300, 8xH100" class="citation-link"><sup>[5]</sup></a>.
At the weight level, community benchmarking site llmrun.dev measures the BF16 checkpoint (560.5B parameters counting embeddings) at 1,233.2 GB of [VRAM](https://gpusmith.com/articles/llm-inference-hardware-sizing-guide) in full precision, dropping to 369.9 GB at a 4-bit Q4\_K\_M quantization, with 481+ GB recommended once KV cache and system overhead are included (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=requires%20369.9%20GB%20of%20VRAM%20at%20Q4_K_M%2C%20or%201233.2%20GB%20at%20BF16)). NVIDIA's official NVFP4 checkpoint measures approximately 352 GB on disk, matching a real-world Reddit hardware audit <a href="https://www.reddit.com/r/LocalLLaMA/comments/1twwycb/nemotron_3_ultra_reality_check_no_onebox_128gb/#:~:text=NVFP4%3A%20~352.4%20GB" title="Highlights: NVFP4: ~352.4 GB" class="citation-link"><sup>[6]</sup></a>. No single consumer or even most professional GPUs can hold these weights; llmrun.dev states plainly that "no single GPU has enough memory" and multi-GPU or cluster configurations are required (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=No%20single%20GPU%20has%20enough%20memory)).
On the cost side, cloud GPU rental prices as of July 2026 range from roughly \$2 to \$7 per GPU per hour for [H100 and H200 units](https://gpusmith.com/articles/gpu-cloud-rental-prices-h100-h200) and \$3.75 to \$10 per GPU per hour for [B200 units](https://gpusmith.com/articles/nvidia-ai-gpu-price-index-trends), depending on provider and commitment level <a href="https://getdeploying.com/gpus/nvidia-h100#:~:text=H100%20pricing%20currently%20ranges%20from%20%240.34%2Fhr%20to%20%2414.90%2Fhr%20per%20GPU" title="Highlights: H100 pricing currently ranges from $0.34/hr to $14.90/hr per GPU" class="citation-link"><sup>[7]</sup></a> <a href="https://getdeploying.com/gpus/nvidia-b200#:~:text=B200%20pricing%20currently%20ranges%20from%20%242.69%2Fhr%20to%20%2416.11%2Fhr%20per%20GPU" title="Highlights: B200 pricing currently ranges from $2.69/hr to $16.11/hr per GPU" class="citation-link"><sup>[8]</sup></a>. Using NVIDIA's own published throughput benchmarks of 310.8 tokens per second per GPU on a 4x B200 agentic workload <a href="https://docs.nvidia.com/dynamo/dev/recipes/nemotron-3-ultra#:~:text=the%20B200%20agentic%20target%20measured%20310.8%20system%20output%20tok%2Fs%20per%20GPU%20on%20its%20trace" title="Highlights: the B200 agentic target measured 310.8 system output tok/s per GPU on its trace" class="citation-link"><sup>[9]</sup></a> combined with Lambda's published four-GPU B200 SXM6 pricing, this report calculates an illustrative [self-hosted compute cost](https://gpusmith.com/articles/own-vs-rent-gpus-tco-comparison) near \$6 per million output tokens at full utilization, roughly 2.7 times the [\$2.20 per million output tokens](https://gpusmith.com/articles/llm-cost-per-million-tokens-benchmark) that DeepInfra lists on OpenRouter for the same model <a href="https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b#:~:text=%240.50%20%2F%20%242.20" title="Highlights: $0.50 / $2.20" class="citation-link"><sup>[10]</sup></a>. For local and workstation use, NVIDIA's own deployment cookbook recommends a four-node **[DGX Spark](https://gpusmith.com/articles/nvidia-dgx-spark-review-specs-performance)** cluster (each node \$4,699, 128 GB unified memory) or a single **DGX Station** (starting around \$94,000, 748 GB of coherent GPU-plus-CPU memory) as the only sanctioned non-datacenter paths <a href="https://marketplace.nvidia.com/en-us/enterprise/personal-ai-supercomputers/dgx-spark/#:~:text=%244%2C699.00" title="Highlights: $4,699.00" class="citation-link"><sup>[11]</sup></a> <a href="https://www.exxactcorp.com/Exxact-VWS-158270643-E158270643#:~:text=Starting%20Price%3A%20%2494%2C011.50" title="Highlights: Starting Price: $94,011.50" class="citation-link"><sup>[12]</sup></a>.
This report walks through every documented deployment tier for Nemotron 3 Ultra: local and workstation-class hardware, single-node datacenter GPU configurations, multi-node clusters, and usage-based API access, quantifying VRAM, GPU count, node topology, and dollar cost at each tier and cross-checking NVIDIA's official specifications against independent community benchmarks and cloud pricing data gathered in July 2026.
## Introduction and Background
Nemotron 3 Ultra sits at the top of NVIDIA's third-generation Nemotron family of open-weight language models, released alongside a mid-sized "Super" (120B total, 12B active) and smaller "Nano" (30B total, 3B active) sibling <a href="https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html#:~:text=Cookbooks%20for%20the%20550B%2F55B-active%20hybrid%20Mamba-Transformer%20MoE%20model" title="Highlights: Cookbooks for the 550B/55B-active hybrid Mamba-Transformer MoE model" class="citation-link"><sup>[13]</sup></a>. Unlike a conventional dense transformer, Nemotron 3 Ultra interleaves Mamba-2 state-space layers, Mixture-of-Experts feed-forward blocks, and select attention layers, a design NVIDIA calls a hybrid Mamba-Transformer, further augmented with a Latent Mixture-of-Experts (LatentMoE) routing scheme; NVIDIA's TensorRT-LLM guide notes "Nemotron v3 Ultra additionally uses a Latent Mixture-of-Experts (LatentMoE) design and ships with built-in Multi-Token Prediction (MTP) layers" (<a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=Nemotron%20v3%20Ultra%20additionally%20uses%20a%20Latent%20Mixture-of-Experts%20(LatentMoE" title="Highlights: Nemotron v3 Ultra additionally uses a Latent Mixture-of-Experts (LatentMoE" class="citation-link"><sup>[14]</sup></a>%20design). The model also ships with built-in MTP layers, which let inference engines draft several tokens per forward pass for native speculative decoding, and it was pretrained using an NVFP4 (4-bit floating point) quantization-aware recipe intended to preserve accuracy while cutting memory and compute cost <a href="https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/#:~:text=Includes%20MTP%20layers%20for%20faster%20inference%20through%20native%20speculative%20decoding" title="Highlights: Includes MTP layers for faster inference through native speculative decoding" class="citation-link"><sup>[15]</sup></a>.
The practical consequence of this architecture is that Nemotron 3 Ultra behaves very differently from a same-sized dense model when it comes to hardware planning. Because only 55 billion of its 550 billion total parameters activate per forward pass, its compute cost per token resembles a much smaller model, but its memory footprint, every parameter must still be resident on a GPU or reachable over a fast interconnect, remains that of a genuinely enormous model. NVIDIA's own TensorRT-LLM deployment guide is explicit on this point, noting that developers must "ensure you have sufficient GPU memory for the full parameter weights even though only a fraction of parameters are active per token" <a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=ensure%20you%20have%20sufficient%20GPU%20memory%20for%20the%20full%20parameter%20weights" title="Highlights: ensure you have sufficient GPU memory for the full parameter weights" class="citation-link"><sup>[16]</sup></a>.
NVIDIA's own sources describe pretraining scale in two slightly different ways worth noting honestly rather than glossing over. The official model card states the base checkpoint "was pre-trained for approximately 20T tokens" of crawled and synthetic code, math, science, and general-knowledge data <a href="https://build.nvidia.com/nvidia/nemotron-3-ultra-550b-a55b/modelcard#:~:text=pre-trained%20for%20approximately%2020T%20tokens" title="Highlights: pre-trained for approximately 20T tokens" class="citation-link"><sup>[17]</sup></a>, while NVIDIA's developer blog separately frames Nemotron 3 Ultra as "building on a 10T token pre-training foundation, Nemotron 3 Ultra adds 212B new tokens" targeting legal, wiki-derived factual, and refreshed GitHub data through September 30, 2025 <a href="https://developer.nvidia.com/blog/nvidia-nemotron-3-ultra-powers-faster-more-efficient-reasoning-for-long-running-agents/#:~:text=Building%20on%20a%2010T%20token%20pre-training%20foundation%2C%20Nemotron%203%20Ultra%20adds%20212B%20new%20tokens" title="Highlights: Building on a 10T token pre-training foundation, Nemotron 3 Ultra adds 212B new tokens" class="citation-link"><sup>[18]</sup></a>; this report cannot fully reconcile the "20T total" and "10T foundation plus 212B" framings from public NVIDIA materials alone and presents both figures as published rather than silently picking one. Post-pretraining stages included supervised fine-tuning, reinforcement learning with GRPO, and a novel Multi-Teacher On-Policy Distillation (MOPD) stage that trains the model against feedback from more than ten domain-specialist teacher models. The model, its training data, and its recipes are released under the Linux Foundation's OpenMDW-1.1 license, which NVIDIA frames as a more permissive successor to prior Nemotron license terms <a href="https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/blob/main/README.md#:~:text=OpenMDW%20License%20Agreement%2C%20version%201.1" title="Highlights: OpenMDW License Agreement, version 1.1" class="citation-link"><sup>[19]</sup></a>.
Because the weights, datasets, and fine-tuning recipes are all open, Nemotron 3 Ultra is being evaluated by a wide spectrum of users, from hyperscale cloud providers running it at scale for millions of agentic tool calls to individual hobbyists on Reddit trying to squeeze a runnable quantization onto a single high-memory workstation.
```Those two audiences face almost entirely different hardware and cost questions, and this report addresses both, working from NVIDIA's own model cards and deployment guides outward through independent benchmarks, cloud pricing trackers, and firsthand community reports gathered as of July 2026.
## Local and Workstation-Class Deployment
The honest starting point for anyone evaluating Nemotron 3 Ultra for local use is that, as of July 2026, there is no consumer-grade, single-GPU, or even single-workstation path that NVIDIA officially sanctions for the full model. NVIDIA's own deployment cookbook index lists only two non-datacenter targets for Nemotron 3 Ultra: a clustered **DGX Spark** configuration and a single **DGX Station** <a href="https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html#:~:text=Deploy%20across%20a%204x%20DGX%20Spark%20cluster" title="Highlights: Deploy across a 4x DGX Spark cluster" class="citation-link"><sup>[20]</sup></a>.
**DGX Spark**, NVIDIA's smallest AI desktop appliance, pairs a GB10 Grace Blackwell Superchip with 128 GB of coherent unified memory and roughly 1 petaFLOP of FP4 compute in a 150mm-by-150mm enclosure, retailing at **\$4,699** through NVIDIA's marketplace <a href="https://marketplace.nvidia.com/en-us/enterprise/personal-ai-supercomputers/dgx-spark/#:~:text=%244%2C699.00" title="Highlights: $4,699.00" class="citation-link"><sup>[11]</sup></a>. A single Spark's 128 GB is far short of the roughly 352 GB the NVFP4 checkpoint occupies on disk, so NVIDIA's official cookbook instead documents deployment "across a 4x DGX Spark cluster with vLLM, including tensor parallelism, RoCE networking, MTP speculative decoding, and NVIDIA AIPerf benchmarking," which pools four units into 512 GB of aggregate unified memory <a href="https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html#:~:text=Deploy%20across%20a%204x%20DGX%20Spark%20cluster%20with%20vLLM%2C%20including%20tensor%20parallelism%2C%20RoCE%20networking%2C%20MTP%20speculative%20decoding%2C%20and%20NVIDIA%20AIPerf%20benchmarking" title="Highlights: Deploy across a 4x DGX Spark cluster with vLLM, including tensor parallelism, RoCE networking, MTP speculative decoding, and NVIDIA AIPerf benchmarking" class="citation-link"><sup>[21]</sup></a>. At list price, four Spark units cost **\$18,796** before networking hardware, a meaningful capital outlay but one still well below datacenter GPU pricing.
**DGX Station** takes a different approach: a single deskside unit built around one GB300 Grace Blackwell Ultra Desktop Superchip, offering 252 GB of HBM3e GPU memory plus 496 GB of LPDDR5X CPU memory bridged by NVLink-C2C at 900 GB/s, for **748 GB of coherent memory** in total, enough to hold the NVFP4 checkpoint's roughly 352 GB of weights with substantial headroom <a href="https://www.nvidia.com/en-us/products/workstations/dgx-station/#:~:text=748%20GB%20of%20coherent%20memory" title="Highlights: 748 GB of coherent memory" class="citation-link"><sup>[22]</sup></a>. Because the GPU's own HBM3e (252 GB) alone is smaller than the model, NVIDIA's cookbook for this path explicitly relies on "coherent CPU memory for selective MoE expert offloading and a FlashInfer TensorRT-LLM NVFP4 backend," meaning inactive experts are held in system memory and pulled across the NVLink-C2C interconnect on demand rather than all 550 billion parameters residing in GPU memory simultaneously <a href="https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html#:~:text=Deploy%20on%20a%20single%20GB300-based%20DGX%20Station%20with%20vLLM%2C%20using%20coherent%20CPU%20memory%20for%20selective%20MoE%20expert%20offloading" title="Highlights: Deploy on a single GB300-based DGX Station with vLLM, using coherent CPU memory for selective MoE expert offloading" class="citation-link"><sup>[23]</sup></a>. System integrator Exxact lists a DGX Station starting at **\$94,011.50**, and other OEM channel partners have listed configurations from roughly \$97,000 to \$99,500 <a href="https://www.exxactcorp.com/Exxact-VWS-158270643-E158270643#:~:text=Starting%20Price%3A%20%2494%2C011.50" title="Highlights: Starting Price: $94,011.50" class="citation-link"><sup>[12]</sup></a> <a href="https://www.servethehome.com/nvidia-dgx-station-systems-available-at-last-gb300-gb200-workstations-for-your-desktop/#:~:text=It%20is%20highly%20unlikely%20these%20will%20sell%20for%20under%20%2480-85K" title="Highlights: It is highly unlikely these will sell for under $80-85K" class="citation-link"><sup>[24]</sup></a>.
Below the DGX tier, the model becomes genuinely impractical. Community quantizer Unsloth published dynamic GGUF quantizations for llama.cpp-family runtimes with the guidance to "run 2-bit on 200GB RAM, 3-bit on 256GB, 8-bit on 600GB," while its maintainer noted on Reddit that the most aggressive quantization the team could produce without breaking usability was "190GB 1-bit dynamic" <a href="https://www.reddit.com/r/unsloth/comments/1twsd68/nvidia_nemotron_3_ultra_is_out_now/#:~:text=Run%202-bit%20on%20200GB%20RAM%2C%203-bit%20on%20256GB%2C%208-bit%20on%20600GB" title="Highlights: Run 2-bit on 200GB RAM, 3-bit on 256GB, 8-bit on 600GB" class="citation-link"><sup>[25]</sup></a> <a href="https://www.reddit.com/r/unsloth/comments/1twsd68/nvidia_nemotron_3_ultra_is_out_now/#:~:text=lowest%20we%20could%20push%20to%20was%20190GB%201-bit%20dynamic" title="Highlights: lowest we could push to was 190GB 1-bit dynamic" class="citation-link"><sup>[26]</sup></a>. A separate independent hardware audit on r/LocalLLaMA testing a 128 GB AMD Strix Halo (Ryzen AI MAX+ 395) unified-memory mini PC found "I did not find a direct GGUF / llama.cpp route for Ultra during this scan," concluding the full model is a "watchlist item" rather than a runnable target on that class of hardware, while the mid-sized Nemotron 3 Super (120B total, 12B active) did run directly at roughly 17.9 tokens per second <a href="https://www.reddit.com/r/LocalLLaMA/comments/1twwycb/nemotron_3_ultra_reality_check_no_onebox_128gb/#:~:text=I%20did%20not%20find%20a%20direct%20GGUF%20%2F%20llama.cpp%20route%20for%20Ultra%20during%20this%20scan" title="Highlights: I did not find a direct GGUF / llama.cpp route for Ultra during this scan" class="citation-link"><sup>[27]</sup></a>. One commenter on that thread summarized the consensus bluntly: "Ultra-size models are not meant to run on 128GB devices" <a href="https://www.reddit.com/r/LocalLLaMA/comments/1twwycb/nemotron_3_ultra_reality_check_no_onebox_128gb/#:~:text=Ultra-size%20models%20are%20not%20meant%20to%20run%20on%20128GB%20devices" title="Highlights: Ultra-size models are not meant to run on 128GB devices" class="citation-link"><sup>[28]</sup></a>. Other users estimated a workable 4-bit build could be assembled for roughly \$2,000 to \$2,500 in used server hardware, accepting throughput "around 10 tokens per second," a workable but far from snappy interactive pace <a href="https://www.reddit.com/r/unsloth/comments/1twsd68/nvidia_nemotron_3_ultra_is_out_now/#:~:text=You%20can%20run%204%20bit%20for%202-2.5k%20if%20you%27re%20willing%20to%20accept" title="Highlights: You can run 4 bit for 2-2.5k if you're willing to accept" class="citation-link"><sup>[29]</sup></a>. A different commenter on the same thread called the 3-bit path "almost no one owns and with effectively zero context," a pointed reminder that quantized VRAM figures alone do not guarantee a usable context window <a href="https://www.reddit.com/r/unsloth/comments/1twsd68/nvidia_nemotron_3_ultra_is_out_now/#:~:text=almost%20no%20one%20owns%20and%20with%20effectively%20zero%20context" title="Highlights: almost no one owns and with effectively zero context" class="citation-link"><sup>[30]</sup></a>.
Despite the hardware barrier, community interest in local quantizations has been substantial. llmrun.dev's tracker shows the official BF16 checkpoint has accumulated "67.2K downloads" and "16.6K quant downloads," with independent GGUF conversions available from repositories including unsloth, AesSedai, and ddh0, plus a 4-bit MLX-format build for Apple Silicon from mlx-community (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=67.2K%20downloads)). That level of community tooling activity, despite the model's impracticality on nearly all locally available hardware, mirrors the broader pattern seen with the Nemotron 3 Super and Nano siblings, which NVIDIA positions as the more deliberately local-friendly members of the family.
_Table 1_ below breaks out the full community GGUF quantization ladder for the BF16 checkpoint as measured by llmrun.dev, illustrating just how much precision reduction is required before the model becomes even theoretically local, and confirming that even the most aggressive practical quantization remains far beyond consumer GPU memory.
**Table 1. Community GGUF quantization tiers for the BF16 checkpoint (llmrun.dev estimates)**
| Quantization | VRAM Required | Downloadable File Size | Community Assessment |
| --- | --- | --- | --- |
| **IQ2\_XXS** (smallest practical tier) | 169.6 GB | 154.14 GB | Still "exceeds the unified memory of most consumer Macs" (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=exceeds%20the%20unified%20memory%20of%20most%20consumer%20Macs)) |
| **Q4\_K\_M** (recommended default) | 369.9 GB | 336.31 GB | "Offers the best balance of quality and VRAM usage" (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=offers%20the%20best%20balance%20of%20quality%20and%20VRAM%20usage)) |
| **Q5\_K\_S** (higher quality) | 423.9 GB | not separately published | "Provides better quality if you have the VRAM" (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=provides%20better%20quality%20if%20you%20have%20the%20VRAM)) |
| **BF16** (full precision) | 1,233.2 GB | 1,121.05 GB | Full-precision baseline; "the full-precision BF16 version is 1121.05 GB" (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=the%20full-precision%20BF16%20version%20is%201121.05%20GB)) |
No tier in _Table 1_ fits inside a single consumer or most professional GPUs. llmrun.dev's own compatibility checker states flatly that "no single consumer GPU has enough VRAM to run NVIDIA Nemotron 3 Ultra 550B A55B BF16 at Q4\_K\_M (369.9 GB)" and that "multi-GPU or professional hardware is required" for any of the tiers above (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=No%20single%20consumer%20GPU%20has%20enough%20VRAM%20to%20run%20NVIDIA%20Nemotron%203%20Ultra%20550B%20A55B%20BF16%20at%20Q4_K_M)).
## Single-Node Datacenter Deployment: Minimum GPU Requirements by Platform
For anyone deploying inside a datacenter or cloud GPU instance, NVIDIA publishes concrete minimums that differ by both GPU generation and checkpoint precision. Precision is the single largest lever on hardware requirements: the BF16 checkpoint needs roughly twice the GPU count of the NVFP4 checkpoint at every tier.
For the BF16 checkpoint, NVIDIA's Hugging Face model card lists a minimum of **8x GB200, B200, GB300, or B300; 16x H100; or 8x H200** <a href="https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/blob/main/README.md#:~:text=8x%20GB200%2FB200%2FGB300%2FB300%2C%2016x%20H100%2C%208x%20H200" title="Highlights: 8x GB200/B200/GB300/B300, 16x H100, 8x H200" class="citation-link"><sup>[4]</sup></a>, and NVIDIA's own recommendation frames a single node of **8x B200 ("approximately 1.5 TB aggregate HBM") as the practical minimum, fitting the BF16 weights plus KV cache with headroom, while H100, H200, GB200, and GB300 configurations of eight or more GPUs are described as requiring multi-node orchestration via Ray v2** <a href="https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/blob/main/README.md#:~:text=8%C3%97%20B200%20(%E2%89%881.5%20TB%20aggregate%20HBM" title="Highlights: 8× B200 (≈1.5 TB aggregate HBM" class="citation-link"><sup>[31]</sup></a> <a href="https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/blob/main/README.md#:~:text=%E2%89%A58%20GPUs%20across%20H100%20%2F%20H200%20%2F%20GB200%20%2F%20GB300%2C%20orchestrated%20with%20Ray%20v2" title="Highlights: ≥8 GPUs across H100 / H200 / GB200 / GB300, orchestrated with Ray v2" class="citation-link"><sup>[32]</sup></a>.
For the NVFP4 checkpoint, the requirement drops substantially. NVIDIA's build.nvidia.com model card, mirrored on the equivalent docs.api.nvidia.com reference page, lists a minimum of **4x GB200, 4x B200, 4x GB300, 4x B300, or 8x H100** <a href="https://build.nvidia.com/nvidia/nemotron-3-ultra-550b-a55b/modelcard#:~:text=4xGB200%2C%204xB200%2C%204x%20GB300%2C%204x%20B300%2C%208xH100" title="Highlights: 4xGB200, 4xB200, 4x GB300, 4x B300, 8xH100" class="citation-link"><sup>[5]</sup></a> <a href="https://docs.api.nvidia.com/nim/reference/nvidia-nemotron-3-ultra-550b-a55b#:~:text=4xGB200%2C%204xB200%2C%204x%20GB300%2C%204x%20B300%2C%208xH100" title="Highlights: 4xGB200, 4xB200, 4x GB300, 4x B300, 8xH100" class="citation-link"><sup>[33]</sup></a>, and its quick-start guide recommends "single-node: 4x B200 (fits NVFP4 weights plus KV cache with headroom)" as the practical baseline <a href="https://build.nvidia.com/nvidia/nemotron-3-ultra-550b-a55b/modelcard#:~:text=4%C3%97%20B200%20(fits%20NVFP4%20weights%20plus%20KV%20cache%20with%20headroom" title="Highlights: 4× B200 (fits NVFP4 weights plus KV cache with headroom" class="citation-link"><sup>[34]</sup></a>). NVIDIA's TensorRT-LLM deployment guide corroborates this and explains why H100 needs twice as many GPUs as Blackwell-class chips for the same checkpoint: Hopper "lacks a native NVFP4 tensor-core GEMM," so the NVFP4 weights are dequantized on the fly through a W4A16 fallback path, which "requires a minimum of 8x H100 (fewer may suffice on the higher-memory H200) and delivers somewhat lower throughput than Blackwell" <a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=lacks%20a%20native%20NVFP4%20tensor-core%20GEMM" title="Highlights: lacks a native NVFP4 tensor-core GEMM" class="citation-link"><sup>[35]</sup></a> (<a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=requires%20a%20minimum%20of%208x%20H100%20(fewer%20may%20suffice%20on%20the%20higher-memory%20H200" title="Highlights: requires a minimum of 8x H100 (fewer may suffice on the higher-memory H200" class="citation-link"><sup>[36]</sup></a>%20and%20delivers%20somewhat%20lower%20throughput%20than%20Blackwell).
NVIDIA's NIM (NVIDIA Inference Microservices) support matrix goes a level deeper, publishing the exact tensor-parallel (TP) and pipeline-parallel (PP) topology validated for every supported GPU and precision combination. On B200, for instance, "BF16 TP=8, PP=1 on a single node (8 GPUs); NVFP4 TP=2, TP=4, or TP=8 on a single node" are all validated configurations, while H100 in BF16 requires "TP=8, PP=2 across two nodes (16 GPUs total)" with inter-node InfiniBand or RoCE networking, and GB200 in BF16 needs "TP=4, PP=2 across two NVL trays of four GPUs" <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/support-matrix-day-0.html#:~:text=BF16%20TP%3D8%2C%20PP%3D1%20on%20a%20single%20node%20(8%20GPUs" title="Highlights: BF16 TP=8, PP=1 on a single node (8 GPUs" class="citation-link"><sup>[37]</sup></a>%3B%20NVFP4%20TP%3D2%2C%20TP%3D4%2C%20or%20TP%3D8%20on%20a%20single%20node) <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/support-matrix-day-0.html#:~:text=BF16%20TP%3D8%2C%20PP%3D2%20across%20two%20nodes%20(16%20GPUs%20total" title="Highlights: BF16 TP=8, PP=2 across two nodes (16 GPUs total" class="citation-link"><sup>[38]</sup></a> <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/support-matrix-day-0.html#:~:text=BF16%20TP%3D4%2C%20PP%3D2%20across%20two%20NVL%20trays%20of%20four%20GPUs" title="Highlights: BF16 TP=4, PP=2 across two NVL trays of four GPUs" class="citation-link"><sup>[39]</sup></a>. _Table 2_ below consolidates these NVIDIA-published minimums across GPU platforms and both checkpoints.
**Table 2. Minimum validated GPU count by platform and precision (NVIDIA official specifications)**
| GPU Platform | Per-GPU Memory | BF16 Minimum GPUs | NVFP4 Minimum GPUs | Topology Notes |
| --- | --- | --- | --- | --- |
| **H100** (Hopper) | 80 GB HBM3 | 16 (TP8, PP2, 2 nodes) | 8 (TP8, PP1, single node, W4A16 fallback) | Hopper lacks native NVFP4 tensor cores; runs dequantized <a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=lacks%20a%20native%20NVFP4%20tensor-core%20GEMM" title="Highlights: lacks a native NVFP4 tensor-core GEMM" class="citation-link"><sup>[35]</sup></a> |
| **H200** | 141 GB HBM3e | 8 (TP8, PP1, single node) | 4 or 8 (TP4 or TP8, single node) | Higher memory than H100 may allow fewer GPUs for NVFP4 (<a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/support-matrix-day-0.html#:~:text=NVIDIA-H200%22%3A%20BF16%20TP%3D8%2C%20PP%3D1%20on%20a%20single%20node%20(8%20GPUs" title="Highlights: NVIDIA-H200": BF16 TP=8, PP=1 on a single node (8 GPUs" class="citation-link"><sup>[40]</sup></a>%3B%20NVFP4%20TP%3D4%20or%20TP%3D8%2C%20PP%3D1%20on%20a%20single%20node) |
| **B200** | 180 to 192 GB HBM3e | 8 (TP8, PP1, single node) | 4 (recommended minimum; 2 or 8 also validated) | NVIDIA's own recommended baseline is 4x B200; NVFP4 TP=2, TP=4, or TP=8 are all validated on a single node (<a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/support-matrix-day-0.html#:~:text=BF16%20TP%3D8%2C%20PP%3D1%20on%20a%20single%20node%20(8%20GPUs" title="Highlights: BF16 TP=8, PP=1 on a single node (8 GPUs" class="citation-link"><sup>[37]</sup></a>%3B%20NVFP4%20TP%3D2%2C%20TP%3D4%2C%20or%20TP%3D8%20on%20a%20single%20node) |
| **B300** | 288 GB HBM3e | 4 or 8 (single node) | 4 (recommended minimum; 2 or 8 also validated) | Matches B200 topology at higher per-GPU memory <a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=4x%20B300" title="Highlights: 4x B300" class="citation-link"><sup>[41]</sup></a> |
| **GB200** (Grace Blackwell) | 192 GB per GPU | 8 (TP4, PP2, 2 NVL trays) | 2 or 4 (single node) | Superchip pairs Grace CPU with Blackwell GPUs <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/support-matrix-day-0.html#:~:text=NVIDIA-GB200" title="Highlights: NVIDIA-GB200" class="citation-link"><sup>[42]</sup></a> |
| **GB300** (Grace Blackwell Ultra) | 288 GB per GPU | 4 (single node) | 2 or 4 (single node) | Same silicon family as DGX Station's GPU <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/support-matrix-day-0.html#:~:text=NVIDIA-GB300%22%3A%20BF16%20TP%3D4%20on%20a%20single%20node%3B%20NVFP4%20TP%3D2%20or%20TP%3D4%20on%20a%20single%20node" title="Highlights: NVIDIA-GB300": BF16 TP=4 on a single node; NVFP4 TP=2 or TP=4 on a single node" class="citation-link"><sup>[43]</sup></a> |
_Table 2_ makes the core planning tradeoff visible at a glance: choosing NVFP4 over BF16 roughly halves GPU count at every tier, and choosing Blackwell-class silicon (B200, B300, GB200, GB300) over Hopper (H100, H200) roughly halves it again for the same precision, since Hopper lacks native 4-bit tensor cores. NVIDIA's own storage guidance for the NIM container reinforces the scale involved even before considering compute: "the container image is approximately 38 GB, and the model cache ranges from approximately 330 GB for an NVFP4 profile to approximately 1.1 to 1.7 TB for a BF16 profile," meaning fast local NVMe storage sized in the multiple-terabyte range is itself a deployment prerequisite <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/get-started-nemotron-3-ultra.html#:~:text=the%20container%20image%20is%20approximately%2038%20GB%2C%20and%20the%20model%20cache%20ranges%20from%20approximately%20330%20GB%20for%20an%20NVFP4%20profile%20to%20approximately%201.1%E2%80%931.7%20TB%20for%20a%20BF16%20profile" title="Highlights: the container image is approximately 38 GB, and the model cache ranges from approximately 330 GB for an NVFP4 profile to approximately 1.1–1.7 TB for a BF16 profile" class="citation-link"><sup>[44]</sup></a>.
Independent community measurement broadly confirms NVIDIA's figures from the weights side. Model-hardware tracker llmrun.dev calculates that the 560.5-billion-parameter BF16 checkpoint "requires 369.9 GB of VRAM to load the model weights" at a 4-bit Q4\_K\_M community quantization, and recommends "481+ GB" once KV cache and framework overhead are added, concluding flatly that "no single GPU has enough memory" and multi-GPU or cluster setups are mandatory (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=requires%20369.9%20GB%20of%20VRAM%20to%20load%20the%20model%20weights)) (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=For%20comfortable%20inference%20with%20headroom%20for%20KV%20cache%20and%20system%20overhead%2C%20481%2B%20GB%20is%20recommended)) (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=No%20single%20GPU%20has%20enough%20memory)). The same tracker notes that only three known systems, the NVIDIA DGX H100, NVIDIA DGX A100 (640 GB configuration), and a 512 GB Mac Studio (M3 Ultra), have enough unified or aggregate memory to run even that reduced quantization, and estimates DGX H100 throughput around 47 tokens per second for the 4-bit build (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=3%20devices%20with%20unified%20memory%20can%20run%20NVIDIA%20Nemotron%203%20Ultra%20550B%20A55B%20BF16)).
## Multi-Node and Enterprise-Scale Deployment
Beyond a single 4x or 8x GPU node, enterprises running Nemotron 3 Ultra at scale, or fine-tuning it, typically move to multi-node clusters. NVIDIA's model card recommends Ray v2 as the multi-processing backend for these configurations, with the vLLM serving example spanning "2x 4x GB300" nodes at tensor-parallel size 4 and pipeline-parallel size 2, orchestrated through a `--distributed-executor-backend ray` flag documented identically on NVIDIA's NIM reference page <a href="https://docs.api.nvidia.com/nim/reference/nvidia-nemotron-3-ultra-550b-a55b#:~:text=--distributed-executor-backend%20ray" title="Highlights: --distributed-executor-backend ray" class="citation-link"><sup>[45]</sup></a>. For H100-class hardware, NVIDIA's NIM support matrix documents a 16-GPU, two-node BF16 topology (TP=8, PP=2) requiring "inter-node InfiniBand or RoCE networking" <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/support-matrix-day-0.html#:~:text=Requires%20inter-node%20InfiniBand%20or%20RoCE%20networking" title="Highlights: Requires inter-node InfiniBand or RoCE networking" class="citation-link"><sup>[46]</sup></a>, underscoring that network fabric, not just GPU count, is part of the hardware bill of materials for BF16 deployments on older silicon.
NVIDIA's NIM container also documents per-GPU runtime tuning as a practical node configuration reference. For a 2-GPU B200 or GB200 NVFP4 profile, it recommends capping concurrency at `--max-num-seqs 64`; for the 8-GPU, 2-node H100 BF16 profile, `--max-num-seqs 512`; and for memory-tighter 8-GPU H200 BF16 and 4-GPU B300/GB300 BF16 profiles, it recommends lowering `--gpu-memory-utilization` to 0.98 or 0.99 respectively while capping `--max-model-len` at 131,072 tokens, trading part of the model's native 1M-token context ceiling for stability at that GPU count. NVIDIA notes that "profiles not listed in this table start successfully with default settings and do not require additional NIM_PASSTHROUGH_ARGS," meaning these tuned flags apply to the more memory-constrained configurations rather than as a universal requirement <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/get-started-nemotron-3-ultra.html#:~:text=Profiles%20not%20listed%20in%20this%20table%20start%20successfully%20with%20default%20settings%20and%20do%20not%20require%20additional%20NIM_PASSTHROUGH_ARGS" title="Highlights: Profiles not listed in this table start successfully with default settings and do not require additional NIM_PASSTHROUGH_ARGS" class="citation-link"><sup>[47]</sup></a>.
For serving software, NVIDIA validates three inference engines for Nemotron 3 Ultra: vLLM, SGLang (recommended container `lmsysorg/sglang:v0.5.11`), and TensorRT-LLM (`nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc21`), each with distinct configuration flags for tensor parallelism, expert parallelism, KV cache dtype, and MTP speculative decoding <a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=nvcr.io%2Fnvidia%2Ftensorrt-llm%2Frelease%3A1.3.0rc21" title="Highlights: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc21" class="citation-link"><sup>[48]</sup></a>. NVIDIA's Dynamo framework additionally supports disaggregated serving, splitting prefill and decode across separate GPU pools; its published disaggregated 1P1D (one prefill, one decode) configuration on 8x B200 total (4 prefill plus 4 decode) measured "61.6 user output tok/s and 231.1 system output tok/s/GPU" on an agentic workload trace at concurrency 32, without MTP enabled <a href="https://docs.nvidia.com/dynamo/dev/recipes/nemotron-3-ultra#:~:text=61.6%20user%20output%20tok%2Fs" title="Highlights: 61.6 user output tok/s" class="citation-link"><sup>[49]</sup></a>.
Enterprises without in-house GPU infrastructure can reach the same topologies through managed cloud services, including Amazon SageMaker JumpStart, Google Cloud, Microsoft Foundry, Oracle Cloud, and a long list of specialized inference providers such as CoreWeave, Together AI, Fireworks AI, and Baseten. AWS's own announcement recommends provisioning "sufficient service quota for GPU instances (for example, ml.p5en.48xlarge, ml.p5.48xlarge, or ml.g7e.48xlarge)" before deployment, and cautions that "GPU instances like ml.p5en.48xlarge can cost several dollars per hour" and will continue billing until the SageMaker endpoint is explicitly deleted <a href="https://aws.amazon.com/blogs/machine-learning/nvidia-nemotron-3-ultra-now-available-on-amazon-sagemaker-jumpstart/#:~:text=Sufficient%20service%20quota%20for%20GPU%20instances%20(for%20example%2C%20ml.p5en.48xlarge%2C%20ml.p5.48xlarge%2C%20or%20ml.g7e.48xlarge" title="Highlights: Sufficient service quota for GPU instances (for example, ml.p5en.48xlarge, ml.p5.48xlarge, or ml.g7e.48xlarge" class="citation-link"><sup>[50]</sup></a>) <a href="https://aws.amazon.com/blogs/machine-learning/nvidia-nemotron-3-ultra-now-available-on-amazon-sagemaker-jumpstart/#:~:text=GPU%20instances%20like%20ml.p5en.48xlarge%20can%20cost%20several%20dollars%20per%20hour" title="Highlights: GPU instances like ml.p5en.48xlarge can cost several dollars per hour" class="citation-link"><sup>[51]</sup></a>. At the largest scale, GB200 NVL72 rack-scale systems are available directly from cloud providers: CoreWeave lists an on-demand GB200 instance (a four-GPU, two-Superchip unit) at **\$42.00 per hour**, equivalent to \$10.50 per GPU per hour, with the provider's documentation noting "each instance is comprised of 2 GB200 or GB300 Superchips, where each Superchip has 1 Grace CPU and 2 Blackwell GPUs" <a href="https://www.coreweave.com/pricing#:~:text=On-Demand%20Price%3A%20%2442.00%20%2F%20Hour" title="Highlights: On-Demand Price: $42.00 / Hour" class="citation-link"><sup>[52]</sup></a> <a href="https://www.coreweave.com/pricing#:~:text=Each%20instance%20is%20comprised%20of%202%20GB200%20or%20GB300%20Superchips%2C%20where%20each%20Superchip%20has%201%20Grace%20CPU%20and%202%20Blackwell%20GPUs" title="Highlights: Each instance is comprised of 2 GB200 or GB300 Superchips, where each Superchip has 1 Grace CPU and 2 Blackwell GPUs" class="citation-link"><sup>[53]</sup></a>. CoreWeave's comparable eight-GPU HGX H100 node, the older-generation topology H100 requires under NIM's BF16 matrix, lists at **\$49.24 per hour**, or \$6.16 per GPU per hour on-demand <a href="https://www.coreweave.com/pricing#:~:text=On-Demand%20Price%3A%20%2449.24%20%2F%20Hour" title="Highlights: On-Demand Price: $49.24 / Hour" class="citation-link"><sup>[54]</sup></a>.
## API and Usage-Based Access: Inference Cost per Token
Not every organization needs to own or rent raw GPUs. Nemotron 3 Ultra is available as a pay-per-token API through OpenRouter, which aggregates several third-party inference providers. As of mid-July 2026, DeepInfra lists the model at **\$0.50 per million input tokens and \$2.20 per million output tokens**, Together AI at \$0.60 input and \$3.60 output, and Venice at \$0.625 input and \$3.125 output, all per million tokens <a href="https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b#:~:text=%240.50%20%2F%20%242.20" title="Highlights: $0.50 / $2.20" class="citation-link"><sup>[10]</sup></a>. Once prompt-caching discounts are factored in, OpenRouter's trailing 7-day weighted-average pricing across all providers works out to \$0.308 per million input tokens and \$2.81 per million output tokens, reflecting a DeepInfra cache-hit rate of 49.1 percent and a Venice cache-hit rate of 88.0 percent <a href="https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b#:~:text=Weighted%20Avg%20Input%20Price" title="Highlights: Weighted Avg Input Price" class="citation-link"><sup>[55]</sup></a> <a href="https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b#:~:text=49.1%25" title="Highlights: 49.1%" class="citation-link"><sup>[56]</sup></a>. Best-in-class provider throughput on the aggregator was measured at 159 tokens per second with a 0.49-second median latency to first useful output <a href="https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b#:~:text=159tok%2Fs" title="Highlights: 159tok/s" class="citation-link"><sup>[57]</sup></a>.
It is possible to sanity-check self-hosted compute economics against these API rates using NVIDIA's own published throughput benchmarks. NVIDIA's Dynamo deployment recipe documentation measured a 4x B200 aggregated vLLM deployment (NVFP4 plus FP8 KV cache, MTP with one speculative token) at **310.8 system output tokens per second per GPU** on an agentic workload trace, and 201.4 tokens per second per GPU on a shorter chat-style trace <a href="https://docs.nvidia.com/dynamo/dev/recipes/nemotron-3-ultra#:~:text=the%20B200%20agentic%20target%20measured%20310.8%20system%20output%20tok%2Fs%20per%20GPU%20on%20its%20trace" title="Highlights: the B200 agentic target measured 310.8 system output tok/s per GPU on its trace" class="citation-link"><sup>[9]</sup></a>. Combining that measured throughput with Lambda's published four-GPU B200 SXM6 instance pricing of \$6.79 per GPU per hour, or \$27.16 per hour for the full four-GPU node, yields an aggregate system throughput of 1,243.2 tokens per second and an implied compute-only cost of roughly **\$6.07 per million output tokens** at continuous full utilization; the same node running the lower-throughput chat trace (805.6 tokens per second aggregate) implies roughly **\$9.37 per million tokens** <a href="https://lambda.ai/pricing#:~:text=NVIDIA%20B200%20SXM6" title="Highlights: NVIDIA B200 SXM6" class="citation-link"><sup>[58]</sup></a>. On 8x H200 (CoreWeave on-demand price of \$50.44 per hour for the full node), Dynamo's measured 27.4 tokens per second per GPU on the agentic trace implies roughly **\$63.93 per million tokens**, over an order of magnitude more expensive per token than the B200 configuration, illustrating how strongly GPU generation and quantization affect self-hosted unit economics <a href="https://docs.nvidia.com/dynamo/dev/recipes/nemotron-3-ultra#:~:text=27.4" title="Highlights: 27.4" class="citation-link"><sup>[59]</sup></a> <a href="https://www.coreweave.com/pricing#:~:text=On-Demand%20Price%3A%20%2450.44%20%2F%20Hour" title="Highlights: On-Demand Price: $50.44 / Hour" class="citation-link"><sup>[60]</sup></a>.
_Table 3_ below summarizes these calculations alongside the OpenRouter API rates. All self-hosted figures are illustrative, derived by combining NVIDIA's own published per-GPU-hour throughput benchmarks with the cited hourly rental prices under an assumption of continuous, fully utilized operation; they exclude engineering, redundancy, storage, and networking overhead and should be read as directional rather than as a quote.
**Table 3. Illustrative cost per million output tokens: self-hosted compute versus managed API (July 2026 data)**
| Deployment | Configuration | Measured System Throughput | Node Rental Cost | Implied Cost per 1M Output Tokens |
| --- | --- | --- | --- | --- |
| Self-hosted, agentic workload | 4x B200, NVFP4, MTP | 310.8 tok/s/GPU (1,243.2 tok/s total) | \$27.16/hr <a href="https://lambda.ai/pricing#:~:text=NVIDIA%20B200%20SXM6" title="Highlights: NVIDIA B200 SXM6" class="citation-link"><sup>[58]</sup></a> | approximately \$6.07 |
| Self-hosted, chat workload | 4x B200, NVFP4, MTP | 201.4 tok/s/GPU (805.6 tok/s total) | \$27.16/hr <a href="https://lambda.ai/pricing#:~:text=NVIDIA%20B200%20SXM6" title="Highlights: NVIDIA B200 SXM6" class="citation-link"><sup>[58]</sup></a> | approximately \$9.37 |
| Self-hosted, agentic workload | 8x H200, NVFP4, MTP | 27.4 tok/s/GPU (219.2 tok/s total) | \$50.44/hr <a href="https://www.coreweave.com/pricing#:~:text=On-Demand%20Price%3A%20%2450.44%20%2F%20Hour" title="Highlights: On-Demand Price: $50.44 / Hour" class="citation-link"><sup>[60]</sup></a> | approximately \$63.93 |
| Managed API, list price | DeepInfra via OpenRouter | not applicable, per-token billing | not applicable | \$2.20 (list) <a href="https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b#:~:text=%242.20" title="Highlights: $2.20" class="citation-link"><sup>[61]</sup></a> |
| Managed API, effective average | OpenRouter, all providers, 7-day weighted | not applicable, per-token billing | not applicable | \$2.81 (effective) <a href="https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b#:~:text=%242.81" title="Highlights: $2.81" class="citation-link"><sup>[62]</sup></a> |
The pattern in _Table 3_ is counterintuitive to a common assumption that self-hosting is automatically cheaper than API access at scale. Even in the best-case, highest-throughput, fully-utilized self-hosted scenario (4x B200 on an agentic trace), the implied per-token compute cost still runs more than double the list price DeepInfra charges through OpenRouter, and roughly 2.16 times the platform's own effective blended rate. This gap exists because managed API providers pool demand from many simultaneous customers, achieving batching efficiency and hardware utilization that a single dedicated node running one workload rarely matches outside of a benchmark trace; it narrows or reverses only at high, sustained request volumes, or where data residency, customization, fine-tuning, or air-gapped requirements make self-hosting a functional necessity rather than a pure cost optimization.
## Comparative Context and Market Positioning
Nemotron 3 Ultra's hardware demands should be read against its own performance claims and against sibling and rival models. NVIDIA reports that Nemotron 3 Ultra "achieves 5.9x, 4.8x, and 1.6x higher inference throughput compared to GLM-5.1-754B-A40B, Kimi-K2.6-1T-A32B, and Qwen-3.5-397B-17B respectively" on an 8,000-token input, 64,000-token output benchmark setting, while reaching "on-par accuracies compared to other state-of-the-art open LLMs across a diverse set of benchmarks" <a href="https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/#:~:text=5.9x%2C%204.8x%2C%20and%201.6x%20higher%20inference%20throughput%20compared%20to%20GLM-5.1-754B-A40B%2C%20Kimi-K2.6-1T-A32B%2C%20and%20Qwen-3.5-397B-17B%20respectively" title="Highlights: 5.9x, 4.8x, and 1.6x higher inference throughput compared to GLM-5.1-754B-A40B, Kimi-K2.6-1T-A32B, and Qwen-3.5-397B-17B respectively" class="citation-link"><sup>[63]</sup></a> <a href="https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/#:~:text=on-par%20accuracies%20compared%20to%20other%20state-of-the-art%20open%20LLMs" title="Highlights: on-par accuracies compared to other state-of-the-art open LLMs" class="citation-link"><sup>[64]</sup></a>. NVIDIA's developer blog separately claims a broader "5x higher throughput compared to other open models in its class," attributing this partly to the NVFP4 pretraining recipe, which "delivers up to 5x higher throughput per GPU at the same interactivity compared to BF16 on Blackwell," and partly to the model completing agentic benchmarks like SWE-bench using fewer total tokens, which NVIDIA states "lowers the cost for agentic tasks by up to 30%" relative to token-hungrier competitors <a href="https://developer.nvidia.com/blog/nvidia-nemotron-3-ultra-powers-faster-more-efficient-reasoning-for-long-running-agents/#:~:text=5x%20higher%20throughput%20compared%20to%20other%20open%20models%20in%20its%20class" title="Highlights: 5x higher throughput compared to other open models in its class" class="citation-link"><sup>[65]</sup></a> <a href="https://developer.nvidia.com/blog/nvidia-nemotron-3-ultra-powers-faster-more-efficient-reasoning-for-long-running-agents/#:~:text=up%20to%205x%20higher%20throughput%20per%20GPU%20at%20the%20same%20interactivity%20compared%20to%20BF16%20on%20Blackwell" title="Highlights: up to 5x higher throughput per GPU at the same interactivity compared to BF16 on Blackwell" class="citation-link"><sup>[66]</sup></a> <a href="https://developer.nvidia.com/blog/nvidia-nemotron-3-ultra-powers-faster-more-efficient-reasoning-for-long-running-agents/#:~:text=lowers%20the%20cost%20for%20agentic%20tasks%20by%20up%20to%2030%25" title="Highlights: lowers the cost for agentic tasks by up to 30%" class="citation-link"><sup>[67]</sup></a>. These are vendor-published figures rather than independently reproduced benchmarks and should be read with that caveat. An independent third-party benchmark aggregator tells a more mixed story: llmrun.dev's tracked LiveBench scores place the BF16 checkpoint's coding result at "71.3 (#42/75)," its math result at "54.5 (#72/75)," and its reasoning result at "37.5 (#70/75)" among tracked models, a meaningfully more middling placement than NVIDIA's own "on-par accuracies" framing suggests, and a useful reminder to weigh vendor-selected benchmark comparisons against independent leaderboards (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=LiveBench%20Coding%3A%2071.3%20(%2342%2F75)) (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=LiveBench%20Math%3A%2054.5%20(%2372%2F75)) (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=LiveBench%20Reasoning%3A%2037.5%20(%2370%2F75)).
One recurring theme across community deployment reports is that NVFP4, not BF16, is the practical default. The same checkpoint runs across NVIDIA's Hopper, Blackwell, and even Ampere architectures using "specialized NVFP4 quantization kernels," per NVIDIA's developer blog, meaning a single quantized artifact is portable across a heterogeneous GPU fleet rather than requiring separate builds per architecture <a href="https://developer.nvidia.com/blog/nvidia-nemotron-3-ultra-powers-faster-more-efficient-reasoning-for-long-running-agents/#:~:text=The%20same%20NVFP4%20checkpoint%20runs%20on%20NVIDIA%20Hopper%2C%20NVIDIA%20Blackwell%2C%20and%20Ampere%20GPUs" title="Highlights: The same NVFP4 checkpoint runs on NVIDIA Hopper, NVIDIA Blackwell, and Ampere GPUs" class="citation-link"><sup>[68]</sup></a>. Within the Nemotron 3 family itself, the "Super" variant, 120 billion total parameters with 12 billion active per NVIDIA's own count, offers a middle path for organizations that find Ultra's hardware floor prohibitive <a href="https://build.nvidia.com/nvidia/nemotron-3-super-120b-a12b/modelcard#:~:text=120B%20(12B%20active" title="Highlights: 120B (12B active" class="citation-link"><sup>[69]</sup></a>): NVIDIA's own TensorRT-LLM guide lists Super's BF16 minimum at just 4x H100 or H200, or 2x B200/GB200, and its NVFP4 minimum at 2x H100/H200 or a single B200/GB200 <a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=For%20Super%20BF16%2C%20use%204%20or%20more%20GPUs%20on%20H100%2FH200%3B%20for%20Super%20NVFP4%2C%202%20GPUs%20on%20H100%2FH200%20may%20suffice" title="Highlights: For Super BF16, use 4 or more GPUs on H100/H200; for Super NVFP4, 2 GPUs on H100/H200 may suffice" class="citation-link"><sup>[70]</sup></a>, and Super's official NIM model card lists a minimum requirement of "8x H100-80GB" for its own baseline configuration <a href="https://build.nvidia.com/nvidia/nemotron-3-super-120b-a12b/modelcard#:~:text=8%C3%97%20H100-80GB" title="Highlights: 8× H100-80GB" class="citation-link"><sup>[71]</sup></a>. One Reddit user framed the tradeoff succinctly when discussing enterprise adoption: "Nemotron specifically is for companies to fine tune and train," positioning it less as a chat assistant to run casually and more as a foundation for downstream, infrastructure-backed customization <a href="https://www.reddit.com/r/unsloth/comments/1twsd68/nvidia_nemotron_3_ultra_is_out_now/#:~:text=Nemotron%20specifically%20is%20for%20companies%20to%20fine%20tune%20and%20train" title="Highlights: Nemotron specifically is for companies to fine tune and train" class="citation-link"><sup>[72]</sup></a>.
## Data Analysis and Evidence
Aggregating the cloud GPU rental market provides useful bounds for budgeting a Nemotron 3 Ultra deployment. Tracking service getdeploying.com reports **48 providers offering H100** at prices ranging from \$0.34 to \$14.90 per GPU per hour, averaging \$3.76 per GPU per hour on-demand across 142 tracked on-demand listings <a href="https://getdeploying.com/gpus/nvidia-h100#:~:text=H100%20pricing%20currently%20ranges%20from%20%240.34%2Fhr%20to%20%2414.90%2Fhr%20per%20GPU" title="Highlights: H100 pricing currently ranges from $0.34/hr to $14.90/hr per GPU" class="citation-link"><sup>[7]</sup></a> <a href="https://getdeploying.com/gpus/nvidia-h100#:~:text=On-demand%20%7C%20142%20%7C%20%243.76%2Fhr" title="Highlights: On-demand | 142 | $3.76/hr" class="citation-link"><sup>[73]</sup></a>. The same tracker lists **34 providers for H200**, ranging from \$1.00 to \$13.78 per GPU per hour with an on-demand average of \$4.29 <a href="https://getdeploying.com/gpus/nvidia-h200#:~:text=H200%20pricing%20currently%20ranges%20from%20%241.00%2Fhr%20to%20%2413.78%2Fhr%20per%20GPU" title="Highlights: H200 pricing currently ranges from $1.00/hr to $13.78/hr per GPU" class="citation-link"><sup>[74]</sup></a>, and **25 providers for B200** across 113 tracked listings, ranging from \$2.69 to \$16.11 per GPU per hour with an on-demand average of \$6.97 <a href="https://getdeploying.com/gpus/nvidia-b200#:~:text=B200%20pricing%20currently%20ranges%20from%20%242.69%2Fhr%20to%20%2416.11%2Fhr%20per%20GPU" title="Highlights: B200 pricing currently ranges from $2.69/hr to $16.11/hr per GPU" class="citation-link"><sup>[8]</sup></a> <a href="https://getdeploying.com/gpus/nvidia-b200#:~:text=On-demand%20%7C%2035%20%7C%20%246.97%2Fhr" title="Highlights: On-demand | 35 | $6.97/hr" class="citation-link"><sup>[75]</sup></a>. The newer **B300** tier, tracked across 16 providers, ranges from \$3.27 to \$18.00 per GPU per hour <a href="https://getdeploying.com/gpus/nvidia-b300#:~:text=B300%20pricing%20currently%20ranges%20from%20%243.27%2Fhr%20to%20%2418.00%2Fhr%20per%20GPU" title="Highlights: B300 pricing currently ranges from $3.27/hr to $18.00/hr per GPU" class="citation-link"><sup>[76]</sup></a>. Reserved and spot pricing run meaningfully lower across all four tiers; on H100, for example, spot listings average \$1.68 per GPU per hour against a \$3.76 on-demand average <a href="https://getdeploying.com/gpus/nvidia-h100#:~:text=Spot%20%7C%2038%20%7C%20%241.68%2Fhr" title="Highlights: Spot | 38 | $1.68/hr" class="citation-link"><sup>[77]</sup></a>.
Pricing trends over the trailing twelve months add helpful context for budgeting multi-year deployments. getdeploying.com reports that H100 on-demand pricing "has decreased by about 4% since July 2025, dropping from \$3.89 to \$3.72/hr per GPU," that B200 on-demand pricing "has decreased by about 7% since July 2025, dropping from \$7.69 to \$7.19/hr per GPU," while H200 on-demand pricing "has remained stable, averaging around \$4.36/hr per GPU" over the same window <a href="https://getdeploying.com/gpus/nvidia-h100#:~:text=decreased%20by%20about%204%25%20since%20July%202025%2C%20dropping%20from%20%243.89%20to%20%243.72%2Fhr%20per%20GPU" title="Highlights: decreased by about 4% since July 2025, dropping from $3.89 to $3.72/hr per GPU" class="citation-link"><sup>[78]</sup></a> <a href="https://getdeploying.com/gpus/nvidia-b200#:~:text=decreased%20by%20about%207%25%20since%20July%202025%2C%20dropping%20from%20%247.69%20to%20%247.19%2Fhr%20per%20GPU" title="Highlights: decreased by about 7% since July 2025, dropping from $7.69 to $7.19/hr per GPU" class="citation-link"><sup>[79]</sup></a> <a href="https://getdeploying.com/gpus/nvidia-h200#:~:text=remained%20stable%2C%20averaging%20around%20%244.36%2Fhr%20per%20GPU" title="Highlights: remained stable, averaging around $4.36/hr per GPU" class="citation-link"><sup>[80]</sup></a>. The modest downward drift on Hopper and early Blackwell pricing suggests self-hosted compute costs for Nemotron 3 Ultra are more likely to improve than worsen over a typical multi-year hardware amortization window, though this trend applies to raw GPU rental rather than to the model-specific throughput improvements NVIDIA and third parties continue to publish for newer inference engine releases.
Applying these ranges to the GPU counts in _Table 2_ produces the following approximate hourly cost bands for a single node running Nemotron 3 Ultra continuously. An 8x H100 NVFP4 node (the minimum viable H100 configuration) costs roughly \$16 to \$119 per hour at average on-demand and peak listed prices respectively (8 GPUs times \$3.76 to \$14.90 per hour), broadly consistent with CoreWeave's own listed HGX H100 node price of \$49.24 per hour <a href="https://www.coreweave.com/pricing#:~:text=On-Demand%20Price%3A%20%2449.24%20%2F%20Hour" title="Highlights: On-Demand Price: $49.24 / Hour" class="citation-link"><sup>[54]</sup></a>. A 4x B200 NVFP4 node, NVIDIA's own recommended baseline, costs roughly \$27 to \$64 per hour (4 GPUs times \$6.79 published by Lambda, or up to the \$16.11 peak market rate). Lambda's own multi-GPU "1-Click Cluster" product, aimed at production deployments rather than single-node testing, prices a 16-GPU HGX B200 cluster at \$9.86 per GPU per hour, dropping to \$8.87 per GPU per hour at 256 or more GPUs, reflecting the volume discounts typical of committed-capacity cloud contracts <a href="https://lambda.ai/pricing#:~:text=NVIDIA%20HGX%20B200" title="Highlights: NVIDIA HGX B200" class="citation-link"><sup>[81]</sup></a>.
Community-reported hardware audits add a valuable reality check to vendor specifications, since they surface configurations that technically load the model but perform poorly. The llmrun.dev tracker's benchmark for a Q4\_K\_M quantization on a DGX A100 (640 GB configuration) estimated roughly 29 tokens per second, while a 512 GB Mac Studio (M3 Ultra) managed only around 2 tokens per second at the same quantization, illustrating that raw memory capacity alone does not guarantee usable throughput; memory bandwidth and interconnect matter just as much as total capacity (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=NVIDIA%20DGX%20A100%20640GB~29%20tok%2Fs)) (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=Mac%20Studio%20(M3%20Ultra%2C%20512GB)~2%20tok%2Fs). This mirrors NVIDIA's own benchmark data in _Table 3_, where 8x H200 delivered dramatically lower per-GPU throughput (27.4 tokens per second per GPU on the agentic trace) than 4x B200 (310.8 tokens per second per GPU) despite H200 offering more total GPUs and more total memory, a gap attributable to Blackwell's native NVFP4 tensor cores and tighter tensor-parallel topology at TP4 versus TP8, as detailed in the API and Usage-Based Access section above.
Multi-Token Prediction itself accounts for a meaningful share of Blackwell's throughput advantage, and its benefit scales with GPU generation. On the same 4x B200 agentic configuration, disabling MTP drops system throughput from 310.8 to 175.9 tokens per second per GPU, while on the chat trace it falls more modestly from 201.4 to 181.3 tokens per second per GPU; on 8x H200, the MTP-off penalty is smaller in absolute terms, from 27.4 to 26.5 tokens per second per GPU on the agentic trace and from 46.8 to 43.0 on the chat trace <a href="https://docs.nvidia.com/dynamo/dev/recipes/nemotron-3-ultra#:~:text=175.9" title="Highlights: 175.9" class="citation-link"><sup>[82]</sup></a> <a href="https://docs.nvidia.com/dynamo/dev/recipes/nemotron-3-ultra#:~:text=26.5" title="Highlights: 26.5" class="citation-link"><sup>[83]</sup></a>. The pattern suggests MTP speculative decoding delivers its largest wins on already-fast Blackwell hardware and a smaller, though still positive, boost on Hopper, reinforcing that speculative decoding and quantization are complementary rather than substitute levers for hardware planning.
## Case Studies and Real-World Examples
## Amazon SageMaker JumpStart: Managed Enterprise Deployment
AWS added one-click deployment support for Nemotron 3 Ultra to Amazon SageMaker JumpStart shortly after the model's June 2026 release, targeting enterprises that want managed infrastructure without directly provisioning GPUs. AWS's own guidance requires customers to confirm "sufficient service quota for GPU instances (for example, ml.p5en.48xlarge, ml.p5.48xlarge, or ml.g7e.48xlarge)" before deployment, all of which are 8-GPU H100 or H200-class instances aligned with the single-node minimums NVIDIA specifies <a href="https://aws.amazon.com/blogs/machine-learning/nvidia-nemotron-3-ultra-now-available-on-amazon-sagemaker-jumpstart/#:~:text=Sufficient%20service%20quota%20for%20GPU%20instances%20(for%20example%2C%20ml.p5en.48xlarge%2C%20ml.p5.48xlarge%2C%20or%20ml.g7e.48xlarge" title="Highlights: Sufficient service quota for GPU instances (for example, ml.p5en.48xlarge, ml.p5.48xlarge, or ml.g7e.48xlarge" class="citation-link"><sup>[50]</sup></a>). Because SageMaker bills for endpoint uptime rather than per-token usage, AWS explicitly warns customers to "remember to delete your endpoint when finished to avoid ongoing charges," a reminder that reflects how easily an idle 8-GPU H100 or H200 endpoint, at several dollars per GPU per hour, can accumulate unplanned cost <a href="https://aws.amazon.com/blogs/machine-learning/nvidia-nemotron-3-ultra-now-available-on-amazon-sagemaker-jumpstart/#:~:text=Remember%20to%20delete%20your%20endpoint%20when%20finished%20to%20avoid%20ongoing%20charges" title="Highlights: Remember to delete your endpoint when finished to avoid ongoing charges" class="citation-link"><sup>[84]</sup></a>.
## CoreWeave GB200 NVL72: Rack-Scale Inference Capacity
CoreWeave offers on-demand GB200 NVL72 instances specifically suited to the model's Grace Blackwell-class minimums. Each billing unit pairs two GB200 Superchips, providing four Blackwell GPUs, at \$42.00 per hour on-demand, or \$10.50 per GPU per hour, positioning it between B200 (\$6.97 average on-demand per getdeploying.com) and B300 (\$3.27 to \$18.00 range) in per-GPU cost, while offering the tighter NVLink-C2C coherent CPU-GPU memory architecture that NIM's support matrix validates for both 2-GPU and 4-GPU NVFP4 topologies, per this report's Single-Node Datacenter Deployment section above <a href="https://www.coreweave.com/pricing#:~:text=On-Demand%20Price%3A%20%2442.00%20%2F%20Hour" title="Highlights: On-Demand Price: $42.00 / Hour" class="citation-link"><sup>[52]</sup></a>.
## DGX Spark Four-Node Cluster: The Documented Prosumer Path
NVIDIA's own cookbook repository documents a specific, reproducible path for running Nemotron 3 Ultra outside a traditional datacenter: a four-node DGX Spark cluster connected over RoCE (RDMA over Converged Ethernet) networking, using vLLM with tensor parallelism spread across the cluster and MTP speculative decoding enabled, benchmarked with NVIDIA's own AIPerf tooling <a href="https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html#:~:text=Deploy%20across%20a%204x%20DGX%20Spark%20cluster%20with%20vLLM%2C%20including%20tensor%20parallelism%2C%20RoCE%20networking%2C%20MTP%20speculative%20decoding%2C%20and%20NVIDIA%20AIPerf%20benchmarking" title="Highlights: Deploy across a 4x DGX Spark cluster with vLLM, including tensor parallelism, RoCE networking, MTP speculative decoding, and NVIDIA AIPerf benchmarking" class="citation-link"><sup>[21]</sup></a>. At \$4,699 per unit, the complete four-node hardware cost of \$18,796 sits well below even a single datacenter-grade H100 8-pack purchased outright, though it requires careful RoCE network configuration and accepts throughput well below what a dedicated 4x B200 node delivers. This is a real, NVIDIA-documented deployment target rather than a hypothetical configuration, and it represents the closest thing to a "prosumer" path NVIDIA officially supports for the full Ultra checkpoint.
## DGX Station with MoE Expert Offloading: Single-Box Deployment
A newer entry in NVIDIA's cookbook catalog, published July 6, 2026, documents deploying Nemotron 3 Ultra on a single GB300-based DGX Station using vLLM's coherent CPU memory support for "selective MoE expert offloading" together with a FlashInfer TensorRT-LLM NVFP4 backend <a href="https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html#:~:text=Deploy%20on%20a%20single%20GB300-based%20DGX%20Station%20with%20vLLM%2C%20using%20coherent%20CPU%20memory%20for%20selective%20MoE%20expert%20offloading%20and%20a%20FlashInfer%20TensorRT-LLM%20NVFP4%20backend" title="Highlights: Deploy on a single GB300-based DGX Station with vLLM, using coherent CPU memory for selective MoE expert offloading and a FlashInfer TensorRT-LLM NVFP4 backend" class="citation-link"><sup>[85]</sup></a>. This is architecturally distinct from the multi-GPU node configurations elsewhere in this report: rather than sharding the model's experts across multiple GPUs, DGX Station keeps active experts in its 252 GB of GPU HBM3e and swaps inactive ones in from its 496 GB of CPU memory over a 900 GB/s NVLink-C2C link, a technique made viable specifically because Nemotron 3 Ultra activates only 55 of its 550 billion parameters per token <a href="https://www.nvidia.com/en-us/products/workstations/dgx-station/#:~:text=NVIDIA%20NVLink-C2C%20Interconnect" title="Highlights: NVIDIA NVLink-C2C Interconnect" class="citation-link"><sup>[86]</sup></a>. It represents the single-box, non-clustered alternative to the DGX Spark cluster approach, at roughly five times the capital cost but without the RoCE networking complexity of a multi-node setup.
## Independent Consumer Hardware Testing: The Strix Halo Reality Check
Outside NVIDIA's own reference architectures, independent hobbyists have systematically tested what Nemotron 3 Ultra can and cannot do on affordable unified-memory hardware. One widely discussed Reddit post benchmarked a Beelink GTR9 Pro (AMD Ryzen AI MAX+ 395, 128 GB unified memory) and, after initially misjudging which Nemotron 3 variant was runnable, concluded that the 550-billion-parameter Ultra checkpoint has no direct GGUF or llama.cpp path at that memory tier, while the 120-billion-parameter Super variant ran directly via llama.cpp's Vulkan backend at "292.51 pp512 / 17.94 tg128" tokens per second (prompt processing and generation, respectively) <a href="https://www.reddit.com/r/LocalLLaMA/comments/1twwycb/nemotron_3_ultra_reality_check_no_onebox_128gb/#:~:text=292.51%20pp512%20%2F%2017.94%20tg128" title="Highlights: 292.51 pp512 / 17.94 tg128" class="citation-link"><sup>[87]</sup></a>. The thread's most upvoted correction, from a different commenter, stated plainly that "Ultra-size models are not meant to run on 128GB devices," a conclusion this report's independent research corroborates: even NVIDIA's own smallest officially documented path (the 512 GB aggregate DGX Spark cluster) requires four times that memory footprint <a href="https://www.reddit.com/r/LocalLLaMA/comments/1twwycb/nemotron_3_ultra_reality_check_no_onebox_128gb/#:~:text=Ultra-size%20models%20are%20not%20meant%20to%20run%20on%20128GB%20devices" title="Highlights: Ultra-size models are not meant to run on 128GB devices" class="citation-link"><sup>[28]</sup></a>.
## Implications and Future Directions
The clearest implication of Nemotron 3 Ultra's hardware profile is that NVFP4 quantization has effectively become the default deployment path rather than an optional optimization. Every officially documented hardware minimum this report identified, whether NVIDIA's own model card, its TensorRT-LLM guide, its NIM support matrix, or its DGX Station and DGX Spark cookbooks, treats the NVFP4 checkpoint as the primary target and BF16 as a secondary, roughly-double-the-hardware option intended mainly for research, fine-tuning baselines, and cases where maximum numerical precision is worth the added GPU count <a href="https://build.nvidia.com/nvidia/nemotron-3-ultra-550b-a55b/modelcard#:~:text=The%20Base-BF16%20checkpoint%20is%20the%20pre-training%20checkpoint%20and%20is%20primarily%20intended%20for%20research%20and%20fine-tuning%20rather%20than%20serving" title="Highlights: The Base-BF16 checkpoint is the pre-training checkpoint and is primarily intended for research and fine-tuning rather than serving" class="citation-link"><sup>[88]</sup></a>. As more inference software (vLLM, SGLang, TensorRT-LLM, and Dynamo were all validated with NVFP4 kernels at launch) matures its low-precision kernel support, the gap between BF16 and NVFP4 requirements is unlikely to narrow; if anything, future Nemotron releases seem likely to ship NVFP4-native by default, following the same "pretrained in NVFP4" approach NVIDIA already used for Nemotron 3 Ultra itself rather than quantizing after the fact <a href="https://research.nvidia.com/labs/nemotron/Nemotron-3-Ultra/#:~:text=Pretrained%20in%20NVFP4" title="Highlights: Pretrained in NVFP4" class="citation-link"><sup>[3]</sup></a>.
A second implication concerns the growing bifurcation between datacenter-scale and prosumer-scale hardware paths. NVIDIA's decision to publish official cookbooks for both a four-node DGX Spark cluster and a single DGX Station suggests the company sees meaningful demand for below-hyperscale, above-consumer deployment options, even for a model this large. That said, the community testing summarized in this report's case studies indicates the gap between "documented as possible" and "genuinely convenient" remains wide: a four-node Spark cluster or single Station is a five- or six-figure capital commitment requiring specialized configuration (RoCE networking, MoE expert offloading tuning) that most individual developers and small teams will not undertake. As one Reddit commenter put it while discussing why enterprises might still prefer larger, harder-to-run open models over smaller ones that fit on a single workstation, "they need better models than what will fit on things like 128gb of ram," a framing that suggests managed API access will likely remain the default on-ramp for the majority of Nemotron 3 Ultra's eventual users even as local hardware technically catches up <a href="https://www.reddit.com/r/unsloth/comments/1twsd68/nvidia_nemotron_3_ultra_is_out_now/#:~:text=They%20need%20better%20models%20than%20what%20will%20fit%20on%20things%20like%20128gb%20of%20ram" title="Highlights: They need better models than what will fit on things like 128gb of ram" class="citation-link"><sup>[89]</sup></a>.
Finally, the cost calculations in this report's Table 3 suggest that self-hosted infrastructure investment is best justified by factors other than raw per-token cost, at least at the deployment scales examined. Data residency, regulatory or contractual requirements against sending data to third-party inference providers, the ability to fine-tune or continually post-train the model on proprietary data using NVIDIA's published GRPO and MOPD recipes, and sustained, highly concurrent request volumes that push utilization well above what a single-tenant benchmark trace captures are all more defensible reasons to self-host than an assumption that owning hardware is inherently cheaper. As GPU rental prices and API prices both continue to shift, organizations evaluating Nemotron 3 Ultra should treat the \$6 to \$64 per million token range calculated here as a starting framework to re-run against their own actual concurrency and utilization patterns rather than as a fixed number.
## Frequently Asked Questions (FAQs)
**What is Nemotron 3 Ultra's total parameter count?** Nemotron 3 Ultra has 550 billion total parameters, with 55 billion active per token due to its Mixture-of-Experts architecture <a href="https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/blob/main/README.md#:~:text=550B%20(55B%20active" title="Highlights: 550B (55B active" class="citation-link"><sup>[1]</sup></a>).
**What is the minimum GPU memory required to run Nemotron 3 Ultra?** There is no single-GPU answer; the model requires multi-GPU configurations regardless of quantization. The NVFP4 checkpoint occupies approximately 352 GB on disk and needs a minimum of four Blackwell-class GPUs (B200, B300, GB200, or GB300) or eight H100s; the BF16 checkpoint occupies over 1.1 TB and needs a minimum of eight Blackwell-class GPUs, sixteen H100s, or eight H200s, as detailed in _Table 2_ above <a href="https://docs.api.nvidia.com/nim/reference/nvidia-nemotron-3-ultra-550b-a55b#:~:text=4xGB200%2C%204xB200%2C%204x%20GB300%2C%204x%20B300%2C%208xH100" title="Highlights: 4xGB200, 4xB200, 4x GB300, 4x B300, 8xH100" class="citation-link"><sup>[33]</sup></a> <a href="https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-BF16/blob/main/README.md#:~:text=8x%20GB200%2FB200%2FGB300%2FB300%2C%2016x%20H100%2C%208x%20H200" title="Highlights: 8x GB200/B200/GB300/B300, 16x H100, 8x H200" class="citation-link"><sup>[4]</sup></a>.
**How many GPUs does Nemotron 3 Ultra need?** As few as four B200, B300, GB200, or GB300 GPUs for the recommended NVFP4 single-node deployment, up to sixteen H100s for a multi-node BF16 deployment, per NVIDIA's own published minimums summarized in _Table 2_ of this report.
**Can Nemotron 3 Ultra run on a single GPU or a consumer graphics card?** No. Even the most memory-rich single consumer GPUs on the market in 2026 fall far short; llmrun.dev's compatibility checker confirms an NVIDIA GeForce RTX 5090, with 32 GB of VRAM, "requires at least 169.6 GB at IQ2\_XXS," meaning even the most aggressive practical quantization exceeds consumer GPU memory by more than five times (Source: [llmrun.dev](https://llmrun.dev/model/nvidia-nvidia-nemotron-3-ultra-550b-a55b-bf16#:~:text=requires%20at%20least%20169.6%20GB%20at%20IQ2_XXS)).
**Can Nemotron 3 Ultra run on a DGX Spark or other unified-memory device?** A single DGX Spark (128 GB) cannot; NVIDIA's own cookbook requires a four-node DGX Spark cluster (512 GB aggregate) for the NVFP4 checkpoint <a href="https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html#:~:text=Deploy%20across%20a%204x%20DGX%20Spark%20cluster" title="Highlights: Deploy across a 4x DGX Spark cluster" class="citation-link"><sup>[20]</sup></a>. A single DGX Station (748 GB coherent memory) can, using MoE expert offloading between GPU HBM3e and CPU LPDDR5X memory <a href="https://docs.nvidia.com/nemotron/nightly/usage-cookbook/Nemotron-3-Ultra-Base/README.html#:~:text=Deploy%20on%20a%20single%20GB300-based%20DGX%20Station" title="Highlights: Deploy on a single GB300-based DGX Station" class="citation-link"><sup>[90]</sup></a>.
**Does the 1M-token context window affect hardware requirements?** Yes. NVIDIA's NIM documentation notes the model's native context defaults to 262,144 tokens and that extending to the full 1,048,576-token window "requires significantly more KV cache memory, which reduces the number of requests that can be served concurrently," making effective context length a tunable tradeoff against concurrent throughput on a fixed GPU count <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/get-started-nemotron-3-ultra.html#:~:text=requires%20significantly%20more%20KV%20cache%20memory%2C%20which%20reduces%20the%20number%20of%20requests%20that%20can%20be%20served%20concurrently" title="Highlights: requires significantly more KV cache memory, which reduces the number of requests that can be served concurrently" class="citation-link"><sup>[91]</sup></a>.
**Which inference engines support Nemotron 3 Ultra?** NVIDIA validates vLLM, SGLang, and TensorRT-LLM for Nemotron 3 Ultra, alongside its own Dynamo framework for disaggregated serving; all three general-purpose engines shipped NVFP4 kernel support at the model's June 2026 launch, each with its own tensor-parallel, expert-parallel, and speculative-decoding configuration flags <a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=nvcr.io%2Fnvidia%2Ftensorrt-llm%2Frelease%3A1.3.0rc21" title="Highlights: nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc21" class="citation-link"><sup>[48]</sup></a>.
**How much does it cost to run Nemotron 3 Ultra via API?** As of July 2026, list pricing on OpenRouter starts at \$0.50 per million input tokens and \$2.20 per million output tokens through DeepInfra, with a platform-wide effective average (after caching, trailing 7 days) of roughly \$0.308 input and \$2.81 output per million tokens <a href="https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b#:~:text=%240.50%20%2F%20%242.20" title="Highlights: $0.50 / $2.20" class="citation-link"><sup>[10]</sup></a>.
**What does it cost to self-host Nemotron 3 Ultra?** This report's illustrative calculations, combining NVIDIA's published throughput benchmarks with published cloud GPU rental prices, put compute-only self-hosted costs between roughly \$6 and \$64 per million output tokens depending on GPU generation and workload type, generally higher than managed API pricing unless utilization is sustained and high (see _Table 3_).
**What is the difference in hardware requirements between the BF16 and NVFP4 checkpoints?** NVFP4 roughly halves the minimum GPU count at every tier compared to BF16, because it uses 4-bit rather than 16-bit weights; NVIDIA's TensorRT-LLM guide notes the NVFP4 checkpoint on Hopper GPUs still requires double the GPU count of Blackwell because Hopper lacks native 4-bit tensor cores and must dequantize on the fly <a href="https://nvidia.github.io/TensorRT-LLM/latest/deployment-guide/deployment-guide-for-nemotron-3-on-trtllm.html#:~:text=lacks%20a%20native%20NVFP4%20tensor-core%20GEMM" title="Highlights: lacks a native NVFP4 tensor-core GEMM" class="citation-link"><sup>[35]</sup></a>.
**How much local storage does deploying Nemotron 3 Ultra require?** NVIDIA's NIM documentation states the model cache alone ranges from approximately 330 GB (NVFP4) to 1.1 to 1.7 TB (BF16), on top of a roughly 38 GB container image <a href="https://docs.nvidia.com/nim/large-language-models/2.0.6/day-0/get-started-nemotron-3-ultra.html#:~:text=the%20container%20image%20is%20approximately%2038%20GB" title="Highlights: the container image is approximately 38 GB" class="citation-link"><sup>[92]</sup></a>.
## Conclusion
Nemotron 3 Ultra's hardware requirements scale directly with the choices an organization makes about precision and deployment tier, and this report's central finding is that those choices carry roughly a two-times swing in minimum GPU count and an order-of-magnitude swing in implied per-token compute cost. The NVFP4 checkpoint on modern Blackwell-class GPUs, four B200, B300, GB200, or GB300 units in a single node, represents NVIDIA's own recommended baseline and the most GPU-efficient path to running the full model; the BF16 checkpoint, older Hopper-generation hardware, or higher-GPU-count topologies each add substantial cost and complexity without necessarily improving output quality for most production workloads. For teams without dedicated infrastructure, managed API access through OpenRouter's aggregated providers, AWS SageMaker JumpStart, or the model's other cloud partners remains, based on this report's cost comparison, generally more economical per token than self-hosting unless request volume is both large and sustained. For teams that do need local or on-premises deployment, whether for data residency, fine-tuning, or air-gapped requirements, NVIDIA has published two concrete, tested hardware paths below full datacenter scale: a four-node DGX Spark cluster at roughly \$18,800 in hardware, or a single DGX Station at roughly \$94,000 and up, each suited to slightly different operational tradeoffs between networking complexity and single-box simplicity. Given how quickly cloud GPU pricing, quantization tooling, and NVIDIA's own reference architectures are moving in 2026, teams evaluating Nemotron 3 Ultra should treat every dollar figure in this report as a snapshot to be re-verified against current vendor pricing pages at the time of an actual deployment decision, rather than a permanent benchmark.
External Sources
[7]https://getdeploying.com/gpus/nvidia-h100#:~:H100%20pricing%20currently%20ranges%20from%20%240....
[8]https://getdeploying.com/gpus/nvidia-b200#:~:B200%20pricing%20currently%20ranges%20from%20%242....
[49]https://docs.nvidia.com/dynamo/dev/recipes/nemotron-3-ultra#:~:61.6%20user%20output%20tok%2Fs...
[74]https://getdeploying.com/gpus/nvidia-h200#:~:H200%20pricing%20currently%20ranges%20from%20%241....
[76]https://getdeploying.com/gpus/nvidia-b300#:~:B300%20pricing%20currently%20ranges%20from%20%243....
[78]https://getdeploying.com/gpus/nvidia-h100#:~:decreased%20by%20about%204%25%20since%20July%20202...
[79]https://getdeploying.com/gpus/nvidia-b200#:~:decreased%20by%20about%207%25%20since%20July%20202...
nemotron 3 ultranvidia nemotrongpu requirementsvram requirementsllm hardware requirementsdgx sparkdgx stationnvfp4 quantizationself hosting llm costinference cost per token
About GPUSmith
GPU Smith is an independent engineering firm that specifies, procures, integrates and validates private AI compute infrastructure on Nvidia reference architectures, from a single inference node to multi-megawatt compute halls. Every engagement is delivered against written acceptance criteria and an as-built documentation set, with procurement at a disclosed margin and no reseller quota or cloud of its own. Six disciplines: hardware integration and commissioning; cluster architecture and sizing; inference build-out; serving optimization; datacenter operations; and sovereign/air-gapped systems. Core thesis: at sustained load, the amortized cost of owned hardware falls below per-token cloud and API pricing, and GPU Smith locates that crossover for a defined workload and states build/no-build in writing. Sectors served: government and regulated enterprise (bounded inference), scaling AI teams past the ownership crossover, and investors/operators needing technical due diligence.
DISCLAIMER
This document is provided for informational purposes only. No representations or warranties are made regarding the accuracy, completeness, or reliability of its contents. Any use of this information is at your own risk. GPUSmith shall not be liable for any damages arising from the use of this document. This content may include material generated with assistance from artificial intelligence tools, which may contain errors or inaccuracies. Readers should verify critical information independently. All product names, trademarks, and registered trademarks mentioned are property of their respective owners and are used for identification purposes only. Use of these names does not imply endorsement. This document does not constitute professional or legal advice. For specific guidance related to your needs, please consult qualified professionals.