Back to Articles|Published on October 31, 2026|30 min read

GPUSmith Article

800G Optics Compatibility for GPU Clusters: BOM Worksheet

800G optics compatibility for GPU clusters in 2026: a port-to-port BOM worksheet covering OSFP heat sinks, twin-port 2x400G vs single-port 800G, SR8/DR8/2xFR4 reach, validated lists, spares and acceptance tests.

By GPU Smith

800G Optics Compatibility for GPU Clusters: BOM Worksheet

Summary

  1. 01An 800G label alone does not establish compatibility. Match lane count and rate, protocol, cage and heat-sink geometry, and the validated module and firmware pairing at both ends.
  2. 02Twin-port modules carry two independent 400G ports in one cage. HGX dual-plane and single-plane designs require different purchasing choices, and the single-plane option halves total GPU bandwidth.
  3. 03Flat-top and finned modules require separate SKUs and spare pools. Connector, polarity and reach must also match the selected module rather than its speed label alone.
  4. 04Build the BOM by link class, recording both endpoints, lane mapping, parts, fiber, reach and dated qualification evidence. Keep undocumented pairings on test hold before volume ordering.
  5. 05Acceptance records should cover module recognition, speed, FEC, per-lane optical power, BER and fault isolation. Firmware and NOS changes trigger renewed qualification of affected worksheet rows.
Inside this article
  1. 01Executive Summary
  2. 02Introduction and Background
  3. 03Key Changes in 800G GPU Fabric Optics
  4. 04Validated Lists, Platform Guides and Third-Party Optics
  5. 05The Port-to-Port Bill-of-Materials Worksheet
  6. 06Implementation Considerations and Process Changes
  7. 07Data Analysis and Evidence
  8. 08Implications and Future Directions
  9. 09Frequently Asked Questions (FAQs)
  10. 10Conclusion

Executive Summary

The shortest accurate answer to "800G optics compatibility for GPU clusters" is that an 800G label does not identify a compatible part. Three properties must match at every link end: the lane structure, the cage and heat-sink geometry, and the vendor-validated module and firmware pairing. IEEE 802.3df, approved on 16 February 2024, defines 800 Gb/s Ethernet with a parallel x8 structure [1], and the OSFP MSA (Octal Small Form-factor Pluggable Multi-Source Agreement) describes the electrical interface as eight lanes of 100 Gb/s [2]. InfiniBand XDR reaches the same 800 Gb/s label with 200 Gb/s lanes [3], so two "800G" optics can be electrically different.

NVIDIA's reference architectures separate two purchasing decisions. A ConnectX-8 SuperNIC can run dual-plane networking with 2x400 Gb/s twin-port transceivers or single-plane networking with 1x400 Gb/s single-port OSFP transceivers [4]. For HGX, NVIDIA's compute-fabric table lists 256 node-side, 256 leaf-side and 512 switch-to-switch transceivers at 32 nodes, doubling at 64 and 128 nodes; summing those columns gives 1,024, 2,048 and 4,096 east-west transceivers, which is this report's own calculation [5]. Server vendors confirm the platform view: Lenovo describes the ConnectX-8 8180 as one 800Gb XDR InfiniBand port or two 400 Gb Ethernet ports [6] and ships it with an empty OSFP cage [6].

Mechanical fit is not interchangeable. NVIDIA states that ConnectX-8 OSFP SuperNICs support a riding heat sink (RHS) cage only [7], and the OSFP MSA states that OSFP and OSFP-RHS are not compatible [2]. Spares must be held per shell type, not per speed. Every compatibility entry is also a dated vendor claim: NVIDIA's per-firmware validated lists differ between versions, carry lifecycle states, and state that unqualified third-party devices come with no performance guarantees [8].

This report supplies a port-to-port worksheet recording endpoint models, lane mapping, optic at each end, connector, reach, revision basis and status, with a test hold wherever qualification is not documented. Supply conditions add urgency: Cignal AI reported more than 42 million 400G-and-above datacom modules shipped in 2025 [9], and LightCounting reported in April 2026 that demand was exceeding supply by 30%, while expecting the shortages to go away by the end of 2026 [10]. Qualify two sources per link class early and set a written spare policy.

1,024Report-calculated east-west transceivers for the dual-plane HGX design at 32 nodes, counting both link ends
72Twin-port modules on each side of the NVL72 design per SU
50%Reduction in total GPU bandwidth for the HGX single-plane alternative
17 WModule power limit for the Lenovo ConnectX-8 8180 port

Introduction and Background

This report addresses 800G optical transceiver compatibility for AI clusters and is written for private AI fabric architects, server integrators and purchasing teams who order network-interface-card (NIC) to leaf, leaf to spine, and management or storage links at 400G and 800G. The decision it supports is narrow: which end-to-end optic or cable combination to request, and then qualify, for each link, including breakouts and spares. It is based on vendor documentation and standards material retrieved on 31 October 2026, contains no benchmark results, client experience or measured site energy, and does not claim that any unlisted module is supported.

Four questions drive the worksheet:

  • What does each link connect? Endpoint models, cages and speeds, per link class.
  • What lane structure and medium fit? Twin-port or single-port, and copper, AOC or optics by reach.
  • Which exact part is validated? Module, SKU and firmware pair, with a dated source.
  • How many, and what spares? Counts from nodes, rails and links, plus a per-SKU spare policy.

The background is a change in how fabrics are built. NVIDIA's HGX reference architecture uses scalable units (SU) of four compute nodes, each node with eight ConnectX-8 SuperNICs configured for 2x400 Gb Ethernet breakout [5]. The GB300 NVL72 reference defines an SU as one rack of 18 compute trays [11]. One mistaken assumption about cage height or breakout is repeated at scale.

Two complications shape the problem. First, "800G" describes electrically different devices: an Ethernet port built from eight 100G lanes, a twin-port OSFP that houses two 400G ports in one cage [12], and an XDR port built from 200G lanes. Second, compatibility is published as lists tied to firmware and platform revisions, and the "last updated" stamps on NVIDIA's three ConnectX-8 list pages do not follow firmware order. This report therefore cites each list by URL and version and treats it as a claim made on a date.

GPU Smith publishes this report. It is not a transceiver manufacturer, reseller or cloud provider, and it is not an option in any table here. The worksheet method keeps qualification evidence, vendor claims and purchasing decisions in one auditable record; readers with regulated or air-gapped deployments should confirm it against the platform vendor's current documentation and their integrator's acceptance procedures before ordering.

Key Changes in 800G GPU Fabric Optics

Electrical rate, line rate and label are three different numbers

An "800G" label is a rounded product name. The OSFP MSA defines the 800GAUI-8 electrical interface as eight differential lanes using 112G-PAM4 signaling [2], which results in eight lanes of 100 Gb/s at 53.125 Gbaud [2]. The same specification defines a 1.6T interface on the same eight-lane form factor at 106.25 Gbaud with 200 Gb/s lanes. The figure 106.25 therefore means a symbol rate in one context and, in an IEEE task-force logic proposal, the per-lane rate including forward error correction (FEC) overhead for eight PCS (physical coding sublayer) lanes [13]. Coherent's DR8+ datasheet uses the second sense, describing 8x100G PAM4 retimed 106.25 Gb/s electrical lanes [14]. That proposal specifies Reed-Solomon RS(544,514) FEC [13], but it is a draft-stage source and not published clause text.

So the electrical rate, the optical line rate and the label can each differ. The IEEE objectives define 800 Gb/s PHYs over eight pairs of multimode fiber up to at least 100 m and over eight pairs of single-mode fiber up to at least 500 m [15]. Juniper states the same structure from the host side: eight parallel lanes of 100 Gbps [16]. XDR instead delivers 800 Gb/s per port from 200 Gb/s lanes [3], and the later IEEE P802.3dj project adds an optional four-lane 800 Gb/s interface [17]. A purchasing record must state lane count and lane rate next to the label.

Twin-port 2x400G is not single-port 800G

NVIDIA's LinkX documentation says Quantum-2 and Spectrum-4 cages house two 400 Gb/s ports in a single cage called 2x400G twin-port OSFP. IEEE confirms the underlying flexibility: an eight-lane port can be a single x8 implementation or two four-lane implementations [1], and the OSFP MSA lists PMD configurations of 1x800G, 2x400G, 4x200G or 8x100G. A twin-port module is one physical part carrying two independent 400G ports, while a single-port 800G module presents one port. They are not substitutes on a purchase order.

HGX makes the choice explicit. It describes dual-plane east-west networking with 2x400 Gb/s twin-port transceivers and single-plane networking with 1x400 Gb/s single-port OSFP transceivers [4]. Dual plane breaks each interface into 2x400 Gb/s; single plane is a cost-effective alternative that reduces total GPU bandwidth by 50% [18]; and OSFP modules allow migration between the two. For larger designs the HGX page recommends a dual-ported optic to simplify breakout at the ConnectX-8 OSFP port, and the NVL72 page repeats it [11].

One wording issue deserves a note. The HGX page describes single-port 800 GbE SuperNICs with 2x400Gb breakout, yet the ConnectX-8 manual lists C8180 Ethernet configurations as two 400GbE, four 200GbE or eight 100GbE ports [19], and Lenovo states the 800Gb/s maximum is reached as 2x400Gb through a twin transceiver or splitter cable [6]. No single 800GbE Ethernet port configuration appears in the pages retrieved, so "single-port 800 GbE" is a cage-level description and the switch sees 2x400G.

Heat sinks and cages decide mechanical fit

NVIDIA's portfolio deck defines IHS (integrated heat sink) as finned top, for switches only, and RHS (riding heat sink) as flat top, for ConnectX-7, BlueField-3, DGX and liquid-cooled systems [20], and adds that switch and DGX cages have different OSFP heights [20]. The ConnectX-8 manual says OSFP SuperNICs support an RHS cage only [7]. The OSFP MSA states that OSFP in any variant is not compatible with OSFP-RHS, and Juniper uses ORHS for an OSFP module with a riding heat sink [16]. Coherent's part-number key identifies a closed-heatsink variant [14], and Credo sells an active cable with one OSFP end and one OSFP-RHS end [21]. The OSFP MSA says the integrated-heatsink design offers more than 30 W of thermal performance [22].

Cage family matters as well. NVIDIA states that twin-port and single-port OSFP cages are not compatible with the QSFP line, and that QSFP-DD is used in Ethernet switches only, not in ConnectX-7 or BlueField-3 [12]. The QSFP-DD hardware specification describes eight 100 Gb/s lanes for 800 Gb/s [23] and backward compatibility with QSFP-DD and QSFP28 [23], which says nothing about OSFP. Juniper states an OSFP port supports only OSFP optics and a QSFP-DD port only QSFP-DD optics, yet an OSFP end and a QSFP-DD end can interoperate on one link when the media type matches [16]. Dell's Z9864F-ON uses 64 OSFP112 ports [24], while Marvell's reference 2U design allows OSFP or QSFP-DD800 [25].

The answer to "OSFP versus QSFP-DD for AI clusters" is therefore platform-specific. For ConnectX-8 NICs and SN5600-class switches the cage is OSFP, flat top at the NIC and finned top at the switch. A QSFP-DD optic can sit on the far end only where the peer has a QSFP-DD cage and the media type matches.

Reach class and connector: SR8, DR8 and 2xFR4

Reach class determines fiber, connector and polarity. The 802.3df draft resolution deck describes 800GBASE-VR8 as eight multimode fibers to at least 50 m [26] and 800GBASE-DR8-2 as eight single-mode fibers to at least 2 km [26]; SR8 is the 100 m multimode objective [15] and can function as a dual 400GBASE-SR4 [27]. "2xFR4" is an MSA and vendor name for two 400G FR4 channels in one module, not an IEEE 802.3df PHY name.

Connectors vary within a class. NVIDIA's validated list gives the twin-port multimode module as OSFP with 2xMPO12 APC at 850 nm up to 50 m [8] and the 100 m single-mode twin-port module with 2xMPO12 APC at 1310 nm [8]. The IEEE DR8 proposal instead specifies a single-row sixteen-fiber interface [28], NVIDIA offers a variant with a single 8-channel MPO-16/APC connector [29], and Arista's VR8 uses MPO-16 per its data sheet [30]. Cisco's QSFP-DD800 VR8 uses dual MPO-12 APC [31], and the QSFP-DD specification lists MPO-12 two-row, MPO-16 or dual MPO-12 interfaces [23]. US Conec positions the MTP-16 connector for 16-fiber parallel links with 8 Tx and 8 Rx [32], and Panduit describes the MMC very-small-form-factor connector as three times the density of MPO at one-third the size [33]. Two modules both labelled "DR8" or "VR8" can therefore need different fiber assemblies, and CommScope's guidance for Arista notes some MPO-12 APC applications also work with MPO-8 APC [34].

The 2xFR4 class behaves differently. NVIDIA's MMS4X50-NM is a CWDM4 multiplexed 1310 nm design, so fiber splitters cannot be used [35] and direct links need crossed LC fibers. Cisco's QDD-2X400G-FR4 likewise uses two duplex LC connectors [36].

Polarity and cleanliness apply to every parallel class. APC is needed to meet 802.3df return loss [28] and Cisco notes its DR8-2 module requires APC MPO patch cords [36]. Fluke Networks notes that polarity cannot be changed on APC connectors and that TIA standards allow only one Type B patch cord in a link [37]. For 16-fiber parts, US Conec states the MTP-16 keying prevents mating with standard MPO hardware [32], and SENKO's guidance warns that alignment alone does not guarantee system polarity [38]. Order the polarity method with the trunk, not as a field fix.

InfiniBand NDR and XDR versus 800G Ethernet optics

NVIDIA defines NDR as 400 Gbps from four 100 Gbps lanes, with NDR200 on two lanes [39], and an NDR OSFP connector carries two InfiniBand links. XDR changes the lane rate: NVIDIA's single-port MMS4A20 links ConnectX-8 cards with Quantum-X800 switches [40]. An Ethernet 800G DR8 optic (eight 100G lanes) and an XDR 800G DR4 optic (four 200G lanes) are not interchangeable even when both are single-mode OSFP parts with 500 m class reach.

The ConnectX-8 datasheet separates InfiniBand (one port at 800/400/200/100 Gb/s) from Ethernet (one port at 400/200/100 Gb/s) [41]. Record the protocol (InfiniBand or Ethernet) and the generation (NDR or XDR) on every link, not only the speed.

Cable medium by reach and rack path

Copper, active optical cable (AOC) and pluggable optics trade power against reach. NVIDIA's MCP4Y10 passive twin-port DAC (direct attach copper) comes in 0.5 to 2 m lengths [42], Amphenol states passive OSFP cable reach up to 2 m at 112G per lane (Source: www.mouser.lt), and third-party passive 800G datasheets list 3 m [43] [44]. Arista lists 1 m and 2 m for its 800G twinax [45]; Juniper cites a generic ceiling of about seven meters [46], which conflicts with those datasheet values, so treat it as a ceiling and not a design value.

Active copper extends reach at a power cost. NVIDIA's MCA4J80 ACC draws 1.5 W per end at 3, 4 and 5 m [47]. Arista lists an 800G OSFP active electrical cable (AEC) at 1 to 5 m [30], Credo specifies a typical 10 W dissipation for its AEC [48], and Arista offers pre-terminated AOCs from 1 to 30 m [45]. LightCounting reported that an early AOC sales surge was damped by Nvidia's move toward transceivers [10].

Platform constraints override these ranges. NVIDIA states that DGX H100/H200 chassis losses make DACs and ACCs generally unsuitable, with only one 3-meter RHS linear ACC offered [12]; copper on ConnectX-8 mezzanine platforms requires NVIDIA approval; and Dell's switch sheet supports DAC on all 64 ports in a 45 degree C case where optics are not supported [24]. Use platform-listed copper inside the rack, multimode twin-port optics to about 50 m, and single-mode beyond.

Table 1 below summarizes the NVIDIA module and cable families, with rated power taken from the cited datasheets and not from measurements.

Part familyCage and heat sinkLanes, fiber, connectorReachRated power
MMA4Z00-NS / -NS-FLT twin-port SR8-class 2x400GOSFP; finned for switches, flat top (-FLT) for RHS cages850 nm MMF, 2xMPO12 APC50 m OM415 W stated [49]
MMS4X00-NS / -NS-FLT twin-port DR8-class 2x400GOSFP; finned top required in switches1310 nm SMF, 2xMPO12 APC100 m straight, 50 m with splitters17 W [50]
MMS4X00-NM / -NM-FLT twin-port 500 mOSFP; flat-top version for DGX-H100 [51]1310 nm SMF, 2xMPO12 APC [8]500 m; error-correction scheme may differ [51]Not captured
MMS4X50-NM twin-port 2xFR4OSFP; closed finned topCWDM4 1310 nm, dual duplex LC2 kmNot captured; confirm on the module datasheet
MMS4A20 single-port 800G XDR DR4OSFP RHS flat top4x200G, one MPO-12/APC500 m class18 W max [40]
MMS4X00-NS400 single-port 400GOSFP flat top1310 nm SMF, MPO12 APC100 mNot captured in the sources reviewed
MCP4Y10 passive twin-port DACOSFP to OSFP8 x 100G-PAM4 copper0.5 to 2 mPassive
MCA4J80 twin-port ACCOSFP to OSFP8 x 100G-PAM4 copper3 to 5 m1.5 W per end

The table shows why spares and approvals are tracked per SKU: rated power runs from zero for passive copper to 17 or 18 W for single-mode modules, and shell types split one speed into incompatible parts. The multimode part's 15 W figure also illustrates source variance, since the same datasheet gives different typical and maximum values in its specification table. Record which figure was used and the revision.

Figure 01
HGX dual-plane and single-plane purchasing choices
Dual-plane networking2x400 Gb/s twin-port
  • Breaks each interface into two 400G ports, one per plane.
  • One twin-port physical module carries two independent 400G ports.
Single-plane networking1x400 Gb/s single-port OSFP
  • Uses single-port 400G OSFP transceivers for the HGX single-plane variant.
  • Offers a cost-effective alternative with a 50% reduction in total GPU bandwidth.

Replace "test hold" with a dated qualification link only after the pair passes acceptance.

Validated Lists, Platform Guides and Third-Party Optics

NVIDIA publishes ConnectX-8 validated cable lists per firmware version, and the three versions reviewed differ. The v40.44.0208 page lists four transceivers, including a single-port 400G OSFP multimode module and a QSFP112 single-mode module, and no twin-port 800G transceivers. The v40.47.1026 page adds the twin-port 50 m multimode module, the 100 m and 500 m single-mode twin-port modules, and passive OSFP-to-2xQSFP112 splitters validated at 800(2x400)Gbps. The earlier page carried an active splitter described as IB twin port NDR, so one part number can be described differently across revisions. Lists for newer firmware were not reviewed.

Lifecycle labels matter as much as presence. In the v40.47.1026 list the XDR switch Q3400-RA is at the Prototype lifecycle [8]. The tested-switch section lists the SN5600 with 64 OSFP ports [8] and the NDR InfiniBand QM9700; the SN5610 was not found there, although the HGX hardware page names the SN5610 as offering 64 ports of 800 Gbps while the same architecture's logical table names the SN5600. Confirm the switch model against the exact bill of materials.

The list also has a third-party section. It states that devices not qualified by NVIDIA may be used but that no performance guarantees are provided [8]. Entries include an 800G DR8 OSFP to 2x400G QSFP112 breakout AOC and an active copper equivalent. Presence in that table does not mean a specific pair has been jointly qualified. NVIDIA's SN5000 manual points to a separate interconnect specification for supported switch-side cables and transceivers.

Server vendors publish their own platform views, and they are narrower than the NVIDIA lists. Lenovo's ConnectX-8 8180 guide, updated 7 September 2026, states the adapter has an empty OSFP cage [6] and lists a flat-top multimode twin transceiver for the water-cooled variant; its 8240 guide describes a dual-QSFP112 adapter that supports 400Gb NDR InfiniBand or 400 Gb Ethernet [52], with a flat-top 400GbE multimode transceiver rated to 50 m [52]. The 8180 port is limited to 17 W of module power, and Lenovo lists finned-to-flat-top active copper cables. Lenovo's Hybrid AI 289 guide lists server-side transceivers for ConnectX-8 GPU NICs [53] and gives a port-mapping rule: port 1 on the twin transceivers connected to the ConnectX-8 NICs always goes to Plane 1 [53]. Dell's XE9780 guide lists eight ConnectX-8 OSFP ports on B300 configurations [54], and its optics sheet lists 800G OSFP112 modules with breakout cables of 1 to 4 m [55]. HPE offers a one-port OSFP ConnectX-8 adapter [56], Supermicro lists eight OSFP 800 GbE ports [57], and Gigabyte lists eight OSFP ports at 800 Gb/s XDR InfiniBand or dual 400 Gb/s Ethernet [58]. Lenovo's SR680a V4 server states each OSFP cage enables 800Gb/s InfiniBand or Ethernet [59], while Dell notes a combined maximum of four NICs and DPUs in its XE9780 B300 configurations [54]. None of those pages was found to state a validated server-side optic for every embedded port, so the integrator's bill of materials controls.

Optics sellers make their own claims. NADDOD markets a flat-top 2x400G module as compatible with ConnectX-8 C8180, ConnectX-9 and Rubin NVL8 servers [60] and states finned-top shells serve switches while flat-top variants serve ConnectX-8 system links [61]. FS publishes a finned-top OSFP 800GBASE-DR8 datasheet with 16.5 W maximum power [62]. These are vendor claims, and nothing retrieved shows NVIDIA qualifying them for the pairings described. The worksheet status is "vendor claim, qualification not documented: test hold", with the claim's date and URL.

Cross-vendor guidance points the same way. Cisco's data sheet says its OSFP 800G modules are supported on Cisco switches and routers and defers detail to its compatibility matrix [63]; Cisco states FEC is performed on the host platform [31]. Linear designs add a caveat. Juniper defines linear pluggable optics (LPO) as modules relying on the host for retiming and signal conditioning [64], Broadcom's NIC guide states LPO uses about 40% less power than DSP-based transceivers [65], and it notes the LPO standard was not finalized when written [65]. A module correct for one host can be wrong for another when the DSP location differs, so record the module architecture next to the SKU.

The Port-to-Port Bill-of-Materials Worksheet

Link matrix

The worksheet has one row per link class, never one row per speed. Its columns are endpoint A and B model and cage, native lane mapping, optic or cable at each end, fiber connector and reach, and revision basis with status. Table 2 below fills it for the link classes in NVIDIA's reference architectures. Every status is dated 2026-10-31, and "test hold" means the sources reviewed do not document that exact pairing, so it must be qualified on the bench before volume ordering.

Link classEndpoint A / Endpoint BLane mappingOptic or cable at each endFiber, connector, reachRevision basis and status (as of 2026-10-31)
Compute NIC to leaf, dual-plane (HGX)ConnectX-8 C8180, OSFP RHS cage / SN5600, OSFP finned8 x 100G split into two 400G ports, one per planeTwin-port flat top at NIC; twin-port finned top at switchTwo straight fibers, MPO12 APC; reach by moduleNIC-side modules on FW 40.47.1026 list; reference gives no part number. Test hold on switch-end pairing and NOS release
Compute NIC to leaf (GB300 NVL72)ConnectX-8, 4 per tray, 18 trays / SN5600800G port split to 2x400G, one path per planeTwin-port OSFP SR8, 50 m, both ends [66]MPO/APC to MPO/APC multimodeClass-matching SR8 module listed; reference defers part numbers [66]. Test hold on part choice
Leaf to spine, east-westSN5600 / SN5600, OSFP finnedTwin-port 2x400G per cageHGX: "OSFP, 2x400G transceiver"; NVL72: SR8 50 mSR8, DR8 or 2xFR4 by reachHGX optic type not stated in appendix. Test hold
Compute NIC, single-plane variantConnectX-8 / leaf1x400G, unbrokenSingle-port 400G OSFP, e.g. flat-top DR4-classMPO12 APC, 100 mSwitch-end pairing not documented. Test hold; halves GPU bandwidth
DPU in-band and storageBlueField-3 B3240, dual QSFP112 / SN5600400G per portNVL72: QSFP112 multimode at server, twin-port SR8 at switch; HGX: "QSFP, 400G transceiver"MPO, multimodeBlueField-3 lists outside pages reviewed. Test hold
Storage and customer uplinksSpine OSFP / 100G QSFP28 DR1 hosts2x400G fanned to 8x100GTwin-port 2xDR4 single-mode OSFPMPO/APC to LC one-to-four breakoutPart number not given. Test hold on lane-to-LC mapping
In-rack copper breakoutConnectX-8 C8240, dual QSFP112 [7] / SN5600 OSFP800 as 2x400GMCP7Y10 OSFP to 2xQSFP112 passive splitter1 to 3 m class copperListed on FW 40.47.1026 page
XDR InfiniBand 800GConnectX-8 C8180 XDR [19] / Quantum-X800 Q34004 x 200GMMS4A20 at NIC; twin-port 2xDR4 at switchTwo MPO-12/APC per twin-port moduleQ3400-RA lifecycle Prototype. Test hold

Only the copper breakout row rests on a pair that a validated list names directly; the others rest on a module class plus an architecture statement, and the NVL72 appendix sends buyers to OEM, ODM and NVIDIA support for exact part numbers. Replace "test hold" with a dated qualification link only after the pair passes acceptance.

Counting, spares and power

The count rule is BOM count = links x optics per end + cold spares. For a twin-port design, one NIC-side module and one switch-side module serve two 400G links, so the module count per NIC is one at each end while the fiber count is two. NVIDIA's HGX tables state counts as aggregates across both planes [5]:

  • 32 nodes (8 SUs): 256 node-side + 256 leaf-side and 512 switch-to-switch transceivers; 512 cables per link type.
  • 64 nodes (16 SUs): 512 + 512 compute-to-leaf and 1,024 switch-to-switch transceivers; 1,024 cables per link type.
  • 128 nodes (32 SUs): 1,024 + 1,024 compute-to-leaf and 2,048 switch-to-switch transceivers; 2,048 cables per link type.
  • Per node: eight NIC-side twin-port modules and sixteen compute-to-leaf fibers, since each NIC breaks out to one fiber per plane.
  • NVL72: 144 uplinks at 400G per SU for compute traffic, with 72 twin-port modules on each side and 144 fibers per SU for SuperNIC-to-switch links [66].

(Hypothetical Example) A 32-node HGX east-west fabric needs 256 flat-top NIC-side modules, 256 finned leaf-side modules and 512 finned spine-side modules: 1,024 modules and 1,024 fibers. With an assumed 5% cold-spare policy applied per SKU (a purchaser choice, since the sources reviewed prescribe no ratio), the order becomes 269 flat-top (256 x 1.05 rounded up) and 807 finned modules (768 x 1.05 rounded up), 1,076 in total. Flat-top and finned spares are never pooled because the shells are not interchangeable.

Per-rack optic electrical power is module count times vendor-rated watts, a nameplate figure and not measured site energy. For a GB300 NVL72 rack with 72 NIC-side twin-port multimode modules at the datasheet's stated 15 W, the nameplate figure is 1,080 W, or 1,224 W at the 17 W maximum listed in the same datasheet's specification table. For the 32-node HGX fabric above, 1,024 modules at the 17 W twin-port single-mode rating give 17,408 W as an upper bound, assuming every module is that class, which the reference does not state. Switch thermal budget needs its own check: Dell limits one airflow mode of the Z9864F-ON to 32 ports of 17 W optics at 35 degrees C [24], the SN5600 lists typical power with passive cables of 940 W [67], and the SN5610 page gives a separate typical figure for 64 optical modules [67].

Figure 02
From link definition to a qualified BOM row
  1. 01Identify both endpoints

    Record endpoint models, cages and speeds for each link class.

  2. 02Match lanes and medium

    Choose twin-port or single-port operation, then copper, AOC or optics by reach.

  3. 03Document the validated part

    Record the module, SKU and firmware pair with a dated qualification source.

  4. 04Count modules and spares

    Derive counts from nodes, rails and links, and apply a separate spare policy per SKU.

  5. 05Attach acceptance evidence

    Record recognition, speed, FEC, optical power, BER and fault isolation against the worksheet row.

Replace test hold with a dated qualification link only after the pair passes acceptance.

Keep undocumented exact pairings on test hold and qualify them on the bench before volume ordering.

Implementation Considerations and Process Changes

Bench and field acceptance

Acceptance proves that a pair recognized by the host also carries traffic with margin. Table 3 below lists the evidence to capture, using tooling documented in the sources reviewed.

StepMethodEvidence to record
EEPROM and module recognitionmlxlink module query [68] or ethtool -m; NVIDIA's Cable Validation Tool first checks the module can be read [69]Part number, serial, firmware, CMIS status; unreadable-EEPROM diagnostics [70]
Port speed and modemlxconfig port split on ConnectX-8, which needs a power cycle [19]; nv set interface breakout on the switchActive speed, lanes per port, split names; odd/even port rule [71]
FEC modemlxlink FEC histogram [68]Active FEC; host-side configuration at both ends; RS(544,514) is the FEC that the IEEE proposal applies to the optical link [72]
Optical powerDigital diagnostic monitoring [68]Per-lane Rx within 2 dB of each other [69]; a -40 dBm reading on all lanes means no light
BERmlxlink counters [68]; NVIDIA's InfiniBand guide requires 125 minutes of port-up time before BER is measured [73]Raw and effective BER with uptime; the 2.4 x 10^-4 PMD BER figure for RS(544,514) comes from a task-force baseline [72] and is a reference, not an acceptance limit
Fault isolationModule PRBS (pseudo-random bit sequence) test, CMIS modules only [68]; end-face inspection per IEC 61300-3-35 (Source: webstore.iec.ch)PRBS result, inspection images, A/B swap test

Five operating notes apply to the evidence above:

  • DOM limits: digital optical monitoring readings can be absent or approximate on a given module, so absent or zeroed readings are logged as a limitation, not a failure.
  • CMIS revision: OIF notes the data path state machine registers were introduced in CMIS 4.0 [74], places lane and data path status on page 11h [74] and now lists CMIS Revision 5.4 [75], so record the revision a module reports.
  • Polling: NVIDIA warns that mlxlink is not intended for continuous polling [68]; use fleet telemetry for ongoing monitoring [73].
  • Inspection: a connector's optical performance now takes precedence over a failed visual inspection [76], so record both.
  • BER context: treat published BER figures as references and apply the platform vendor's own thresholds.

Change-control checklist

  • Freeze the identity of each end: record NIC and switch model, part number, firmware and NOS release before ordering, because lists are keyed to firmware.
  • Re-check the list on every firmware change: re-read the validated page for the new version and update each row's date. Treat each firmware change as a trigger to re-qualify every row that depends on the list; the vendor's notes for the new version may change which pairings are validated or carry known issues.
  • Record the lane structure: 8x100G Ethernet, 4x200G XDR or 4x100G NDR on the link record, not only the label.
  • Order shells per SKU: finned and flat-top parts are separate SKUs and separate spare pools.
  • Specify polarity and end face with the trunk: APC ends cannot be re-polarized in the field [37].
  • Mark third-party parts as vendor claims: keep the claim date and URL [8].
  • Plan breakout configuration: include lanes-per-port where a four-lane peer is involved and schedule the power cycle for ConnectX-8 split changes.

Data Analysis and Evidence

The quantitative record falls into counts, power and market context. Each figure below is attributed to its originator, and none is a measurement of a particular site.

Counts from NVIDIA's reference tables:

  • NVIDIA's compute-fabric table gives node-to-leaf (compute and switch side) and switch-to-switch transceiver counts for the dual-plane HGX design; this report's own sums are 1,024, 2,048 and 4,096 east-west transceivers for 32, 64 and 128 nodes, counting both ends of the node-to-leaf links [5].
  • The NVL72 design pairs 72 NIC-side with 72 switch-side twin-port modules per SU.
  • NVL72 leaf switches are intentionally underpopulated, which raises switch counts without changing per-link optics.
  • Broadcom's Tomahawk 5 provides 512 instances of 100G PAM4 SerDes [77], the lane count behind 64 ports of 800G.

Nameplate power (vendor-rated, not measured):

  • The OSFP MSA lists a 33.0 W maximum for OSFP/OSFP800; the QSFP-DD specification says a power class 8 module must be read from CMIS for its actual dissipation [23] and that revision 6.0 raised maximum module dissipation to at least 25 W.
  • Juniper states 800G optics require 16 W to 18 W; NVIDIA rates the single-port 800G XDR MMS4A20 at 18 W max and the twin-port single-mode module at 17 W; Arista rates its DSP-based 800G modules at 16 W [30].
  • LPO figures are lower, with Eoptolink listing 8.5 W for an 800G LPO AOC [78], and Broadcom states goals of sub 11 W for single-mode and sub 10 W for multimode DSP-based 800G modules [79].
  • Chassis typical power with passive cables is 940 W for the SN5600 [67], 747 W for the QM9700 [80] and 2,900 W for the Q3400-RA [81].

Market context from named research firms:

  • Cignal AI: more than 42 million high-speed datacom modules shipped in 2025, up more than 80% over 2024 [9]; 800GbE shipments were the growth driver [9]; almost 10 million 1.6TbE modules were forecast for 2026 [9].
  • LightCounting: 800G shipments to more than double in 2026 [82]; 2026 Ethernet optical transceiver growth expectation raised to 120% in September 2026 [83]; demand exceeding supply by 30% in April 2026, with shortages expected to go away by the end of 2026 [10].
  • Dell'Oro: the 1.6 Tbps ramp is expected to be at least five times faster than the 800 Gbps ramp [84].

These are forecasts, and none measures a particular buyer's lead time.

Figure 03
HGX transceiver counts at 32 nodes, both planesTransceivers
Source: NVIDIA's compute-fabric table

Implications and Future Directions

Three trends will change the worksheet:

  • Higher lane rates: 1.6T on 200G lanes and the IEEE P802.3dj project.
  • New platforms: ConnectX-9 and Rubin-era modules.
  • Linear and co-packaged designs: equalization moves to the host.

First, 1.6T is arriving on the same eight-lane OSFP form factor with 200G lanes, and Dell'Oro's 2Q2026 report said that 1.6 Tbps began sampling with a ramp expected in the second half of 2026, a dated snapshot to re-check against later releases [85]. IEEE P802.3dj is at draft stage, with VIAVI reporting draft D2.4 in March 2026 [86] and the Ethernet Alliance expecting ratification in late 2026 or early 2027 [87]. Second, ConnectX-9 servers pair with an 800G single-port module for XDR or 800GbE [88] and an HGX-specific twin-port module targets Rubin NVL8 [89], so rows built for ConnectX-8 will need new endpoint columns. Third, linear and co-packaged designs shift equalization to the host [90]; the Ethernet Alliance roadmap highlights LPO among its themes [91]. Interoperability events now cover DSP-based and linear optics together [92] [93], and the Ultra Ethernet specification adds a standard link-layer retry that does not remove the need for clean physical links [94].

For procurement, the lasting implication is to treat compatibility as data with a date. The worksheet method here treats qualification as a maintained register: each link class has an owner, a vendor-list citation, a bench result and a renewal date tied to firmware and NOS releases. Nothing in this report guarantees support for any unlisted module, and readers should obtain written confirmation from the platform vendor or integrator for any pairing marked test hold, particularly before committing volume orders or changing firmware across a running fabric.

Frequently Asked Questions (FAQs)

OSFP vs QSFP-DD: which is better for 800G AI clusters?

Neither in general; the endpoint sets the cage. ConnectX-8 OSFP SuperNICs accept only RHS cages, NVIDIA's QSFP-DD use is limited to certain Ethernet switches, and Juniper notes the two port types are not interchangeable.

Is there a trustworthy 800G transceiver compatibility matrix?

The authoritative sources are the platform vendor's per-firmware validated lists, each dated and treated as a claim. Reseller statements are separate vendor claims [60].

How do 800G SR8, DR8 and 2xFR4 differ for GPU fabric?

SR8-class parallel multimode reaches about 50 m in NVIDIA's twin-port module, DR8-class parallel single-mode reaches 100 to 500 m, and 2xFR4 is duplex-LC CWDM4 to 2 km that cannot use splitter fibers.

InfiniBand NDR vs 800G Ethernet optics: what differs?

NDR uses four 100G lanes per 400G link and Ethernet 800G uses eight 100G lanes. Match protocol and generation, not only speed.

What goes into an 800G optics bill of materials for an AI cluster?

Count links by link class, multiply by optics per end, add per-SKU spares and record datasheet watts, as in the worksheet.

How do 800G breakout cables and transceiver ports map?

An 800GbE SN5600-family port splits to 2x400GbE or 4x200GbE without limitation [71]; 8x100GbE on an odd port blocks the adjacent even port. On server-side twin transceivers, Lenovo's guide sends port 1 to Plane 1 [53].

Are third-party 800G optics compatible with NVIDIA switches?

NVIDIA's list says unqualified third-party devices may be used but carry no performance guarantees, so mark them test hold until bench evidence exists.

800G AOC vs DAC vs transceiver for GPU racks?

Use platform-listed copper for short in-rack runs, ACC or AEC to about 5 m [30], and optics beyond; DGX H100/H200 systems have specific limits.

Conclusion

Selecting 800G optics for a GPU cluster is a matching problem across lane structure, cage and heat sink, connector and reach class, and vendor-validated pairing. The worksheet in this report turns that problem into a link-by-link record in which each status is dated and each unsupported assumption is a test hold. The most consequential decisions are made before the purchase order: whether each NIC link is twin-port 2x400G or single-port, which shell type each end needs, which fiber assembly the module requires, and which firmware and validated list the entry rests on.

Acceptance evidence then closes the loop. EEPROM recognition, port speed, FEC mode, per-lane optical power, bit error rate and fault isolation each produce a record that can be attached to the worksheet row, and a change-control routine keeps the record current as firmware, platforms and the 1.6T transition move on. Buyers who treat compatibility as a dated claim, rather than a property of the part, can order with a clear view of what has been documented, what is a vendor claim, and what still needs to be proven on their own bench, and they can show that record to integrators and suppliers when a link needs to be explained.

External Sources (94)

About

GPUSmith

Plan and build private AI compute with GPU Smith. We connect workload requirements with GPU hardware, networking, deployment and operating decisions so your infrastructure fits the work it must perform.

GPU Smith provides independent engineering and research for private AI infrastructure. We help technical buyers and operators reason about GPU workload sizing, hardware selection, procurement, deployment and inference operations, from the compute system to the networking, power and cooling requirements around it.

Start with the workload

Useful infrastructure decisions begin with the models, data, throughput, latency and operating constraints a team actually has. GPU Smith helps connect those requirements with system design and deployment choices. The goal is a privately controlled AI environment whose capacity and operational demands are understood before a purchasing decision.

Hardware and supplier research

Our public reference library covers GPUs, complete systems and networking components. The server-vendor directory and manufacturing research help teams investigate suppliers, compare options and follow sources behind technical claims. We distinguish manufacturer specifications from measured benchmarks and advertised capabilities from tested configurations. Reference pages are research resources, not a live inventory listing or a binding equipment quote.

Deployment and operations

GPU Smith's engineering scope includes infrastructure integration, networking, power, cooling and the practical operation of inference workloads. Our published articles explain the tradeoffs behind deployment, procurement and ongoing operations so teams can ask better questions and document decisions.

Work with GPU Smith

Explore the hardware library, GPU references, networking references and vendor research. Contact GPU Smith with your workloads and deployment constraints to discuss a project and confirm current scope, pricing and availability.

Inclusion of a manufacturer or product in our research does not imply a partnership, certification or endorsement.

Disclaimer

This document is provided for informational purposes only. No representations or warranties are made regarding the accuracy, completeness, or reliability of its contents. Any use of this information is at your own risk. GPUSmith shall not be liable for any damages arising from the use of this document. This content was generated with assistance from artificial intelligence tools, which may contain errors or inaccuracies. Readers should verify critical information independently. All product names, trademarks, and registered trademarks mentioned are property of their respective owners and are used for identification purposes only. Use of these names does not imply endorsement. This document does not constitute professional or legal advice. For specific guidance related to your needs, please consult qualified professionals.