Systems and Networking

A package computes nothing until it is built into a machine, and building the machine is a separate industry from making the package.

How it works

How AI chips are wired together

1. One rack. One AI chip is not enough to train a model. So 72 of them are wired together in one rack, a cabinet the size of a wardrobe, and run as one computer.
2. Copper inside. Inside the rack, copper wires link each tray of chips to the switch trays, which pass data between any two chips. Copper carries a signal only a few meters.
3. Light between. Training a large model takes many racks, too far apart for copper. The links between racks carry the signal as flashes of light in glass fiber.
4. Lasers. At each end of a fiber, a laser turns the electrical signal into flashes of light, and a detector turns them back. That lets thousands of chips in many racks work on one model.

Simplified. The trays follow the layout of an Nvidia NVL72 rack; racks and fibers are not drawn to scale.

An accelerator ships as a module, the chip on a small board with its power supply and memory stacks. Nvidia calls its version SXM, and the open-standards equivalent is the Open Compute Project's accelerator module. Eight bolt onto a baseboard that carries their power and the switch chips, which pass data between any two modules at full speed.

Rack-scale systems drop the baseboard. An Nvidia GB300 NVL72 holds 18 compute trays plus nine switch trays, 72 GPUs in all, and draws up to 142 kW^[1]. Vera Rubin NVL72 uses the same rack footprint^[2]. Inside the rack the trays reach each other over copper. Between neighboring racks, active electrical cables, copper cables sold by firms such as Credo, can carry the signal up to 7 meters^[3]. Longer links run on glass fiber with an optical transceiver at each end, where a laser turns the electrical signal into pulses of light and a detector at the far end turns the pulses back into current.

Every AI cluster runs two networks that are not interchangeable. Scale-up binds anywhere from a few dozen accelerators (72 in an Nvidia NVL72 rack) to thousands (9,216 in a Google Ironwood superpod) tightly enough that they can read each other's memory, at terabytes per second; scale-out joins those groups to one another at hundreds of gigabits per second. A byte is eight bits, so the inner network is tens of times faster than the outer one. A model too big for one accelerator is split across many. The parts that exchange data constantly stay inside the scale-up group, while whole blocks of layers can sit on different groups across the slower scale-out network^[4]. So scale-up bandwidth sets how efficiently a large model trains.

Variants and trade-offs

Scale-up: NVLink, UALink and Google's torus

The designs differ in how many accelerators can share one pool of memory.

• NVLink and NVSwitch. Nvidia's per-GPU bandwidth went from 900 GB/s in NVLink 4 to 1,800 in NVLink 5, and the rack's own network moves 130 TB/s on Blackwell^[5]. Nvidia's Rubin launch release of January 2026 gives NVLink 6 as 3,600 GB/s per GPU and 260 TB/s for a Vera Rubin NVL72 rack^[6]. As of October 2026, Nvidia's NVLink product page still lists preliminary figures of 3,000 GB/s and 216 TB/s^[5].
• UALink. The open specification runs 200 Gb/s per lane, four lanes to a station for 800 Gb/s, and can address up to 1,024 endpoints in one group^[7]. AMD's Helios rack, launched in July 2026, joins 72 GPUs and runs UALink over Ethernet for its scale-up network^[8].
• Google's 3D torus. Ironwood, Google's own TPU accelerator, wires 64 chips into each rack, called a cube, and uses optical circuit switches to join cubes into superpods of 9,216 chips that can bypass a failed cube or link^[9].

NVLink bandwidth per GPU by generationGB/s

NVLink bandwidth per GPU by generation

NVLink 4 (Hopper)	900 GB/s
NVLink 5 (Blackwell)	1,800 GB/s
NVLink 6 (Rubin)	3,600 GB/s

Source: NVIDIA NVLink product page (NVLink 4 and 5); NVIDIA Rubin launch release, January 2026 (NVLink 6)

Scale-out: InfiniBand versus Ethernet

InfiniBand, the specialized network built for supercomputers, was the main scale-out network when AI clusters were small and alike. Ethernet switch capacity grew quickly as Broadcom doubled the bandwidth of the switch chips it sells to all comers every generation, from 25.6 Tbps on one chip in Tomahawk 4^[10] to 51.2 in Tomahawk 5^[11] and 102.4 in Tomahawk 6^[12]. In June 2025 the Ultra Ethernet Consortium published its 1.0 specification, an Ethernet-based stack for AI and supercomputing that covers transport protocols, congestion control and direct memory access^[13].

Broadcom Tomahawk switching capacity on one chipTbps

Broadcom Tomahawk switching capacity on one chip

Tomahawk 4	25.6 Tbps
Tomahawk 5	51.2 Tbps
Tomahawk 6	102.4 Tbps

Source: Broadcom product releases for Tomahawk 4, 5 and 6

Nvidia sells both networks, and its data center revenue reached $89.0 billion in the quarter to July 2026, up 117 percent from a year earlier^[14]. In the quarter before, it credited its data center growth to Blackwell 300 products and to demand for its "InfiniBand, Spectrum-X Ethernet, and NVLink" solutions^[15].

Optics: pluggable, then co-packaged

Almost every link longer than 7 meters is optical, and a cluster needs several transceivers, the modules that turn electricity into light and back, for each accelerator. Their power cost is one reason Broadcom and Nvidia are moving the optics onto the switch package itself. Broadcom's Tomahawk 6 Davisson cuts optical interconnect power by 70 percent, more than 3.5 times lower than traditional pluggable modules^[16]. Nvidia's Spectrum-X Photonics switches, due in 2026, reach 512 ports of 800 gigabits and use four times fewer lasers^[17].

Who makes it

Optical modules are assembled where labor and land are cheap, and the Chinese makers went offshore early. InnoLight has run a Thai plant since 2019 alongside Suzhou and Taiwan^[18], and Eoptolink's Thai subsidiary lists sites in Chonburi and Rayong^[19]. The Federal Communications Commission's July 2026 order bars new authorizations for devices that contain logic-bearing hardware components produced by firms on its Covered List, and commenters in the record named optical transceivers as one such component^[20]. The same order notes that the key players in the optical transceiver market, Coherent and InnoLight among them, do not appear on the Covered List^[20].

The lasers inside those modules come from far fewer firms. Indium phosphide is used in the high-speed laser diodes and photodetectors for optical communications, and China accounted for 70 percent of world indium production in 2025^[21]. In February 2025 China's Ministry of Commerce placed indium under new export restrictions^[21]. In March 2026 Nvidia agreed to invest $2 billion in Lumentum, alongside a multibillion purchase commitment and future capacity access rights for advanced laser components^[22]. It made the same $2 billion investment in Coherent, with a multibillion-dollar purchase commitment for advanced laser and optical networking products^[23].

Two US firms, Broadcom and Marvell, design both custom accelerators and switch silicon. Broadcom sells the Ethernet switch chips other firms build their switches around, along with custom AI accelerators, and its AI semiconductor revenue reached $16.7 billion in the third quarter of fiscal 2026, up 221 percent^[24]. Marvell's five-year agreement with AWS, announced in December 2024, covers custom AI chips alongside the optical and networking parts that go with them^[25].

Much of the physical assembly is done by Taiwanese firms. Foxconn booked NT$8.1 trillion of revenue in 2025, up 18 percent, with cloud and networking among the drivers^[26]. Assembly is moving closer to the customer. In April 2025 Nvidia said it was building supercomputer manufacturing plants in Texas with Foxconn in Houston and Wistron in Dallas, with mass production expected to ramp up in the next 12 to 15 months^[27]. In August 2026 Nvidia said Wistron's new plant in Fort Worth, Texas, was producing its GB300 superchips and Foxconn was building its factory in Houston^[28].

The chokepoint

Several contract manufacturers on three continents assemble racks, and the power shelves, copper busbars, cables and connectors have many suppliers. Rack-scale systems are complex, and in its filing for the quarter to April 2026 Nvidia said that complexity had caused production delays^[15]. Moving Foxconn's share to Quanta and Wistron would likely take several quarters, though no source puts a figure on it.

The parts that cannot be replaced that fast are the components that go into the rack.

• Indium phosphide laser capacity. In March 2026 Nvidia agreed to invest $2 billion in Coherent, alongside a multibillion-dollar purchase commitment and future access and capacity rights for Coherent's advanced laser and optical networking products^[23]. In 2025 China produced 70 percent of the world's indium, which those lasers need, and it put indium under new export restrictions in February 2025^[21].
• Indium phosphide substrates. China added indium phosphide substrates, the wafers the lasers are grown on, to its export control list on 4 February 2025^[29]. In its filing for the quarter to June 2026, AXT, a US-listed maker of these wafers that does all its manufacturing in China, called the permits the most significant challenge it faces and said it could not estimate when it would resume shipping the wafers to the United States^[29].
• NVSwitch. NVLink and NVSwitch are Nvidia's own designs, and UALink is an open standard designed for accelerator communications within a pod^[7]. Since May 2025 Nvidia's NVLink Fusion program has let other firms' custom chips use NVLink, including CPUs from Fujitsu and Qualcomm that connect to Nvidia GPUs^[30].
• High-layer-count circuit boards. The boards, and the low-loss laminate they are built from, are concentrated in Taiwan and Japan; see Substrates and PCBs.

Optical module assembly is concentrated, but where it happens follows cost and policy, and it has moved before, as the Chinese makers' Thai plants show^[18].

Key evaluation criteria

• Scale-up domain size is how many accelerators share one pool of memory: up to 1,024 endpoints in the UALink spec^[7] and 9,216 for an Ironwood superpod^[9].
• Bandwidth per accelerator limits how efficiently one model can be split across chips. NVLink 6 is 3,600 GB/s^[6].
• Picojoules per bit is the energy cost of moving data. Broadcom says traditional pluggable optics consume more power as AI traffic grows, and its co-packaged switch cuts optical interconnect power by 70 percent^[16].
• Serviceability changes with co-packaged optics, which move the optics off plug-in modules and onto the switch package. Broadcom's Davisson switch keeps its lasers in field-replaceable modules^[16].
• Supply concentration is in the components: many firms assemble the modules; far fewer can make the lasers inside them.