The single switch that decides whether 10,000 GPUs can "think as one machine"

One sentence has become an engineering truth of the AI era: "the network is the computer." The reason is in how models are trained — one big model is trained across tens of thousands of GPUs at once, and at every step of training every GPU has to "average together" its own result with all the others (a step called all-reduce) before moving on. That means if the switch is even a little slow or congested, the whole cluster of GPUs — each costing tens of thousands of dollars — has to stop and wait together.

This is why an AI cluster is called "network-bound" — the speed ceiling isn't how fast the compute chips are, it's how fast they can swap data with each other. The numbers make it vivid: a cluster of just 1,000 GPUs needs a non-blocking network — every GPU talking to every GPU (all-to-all) — that can carry more than 400 Tbps combined, with almost no dropped packets and latency held steady at microsecond precision. That kind of job simply didn't exist in pre-AI data-center networks.

The economic consequences follow immediately. The network used to be just a "small add-on" to a data center. But in an AI cluster it's become one of the fastest-growing big-ticket costs. Dell'Oro Group estimates that the money going into switches for the AI back-end network will pass $100 billion a year by 2030, and the market for the "switch silicon" alone — before it's assembled into boxes — was already worth about $3.8 billion in 2025.

The heart of tying 10,000 GPUs into one well-connected conversation is a structure called leaf/spine (or fat-tree) — instead of running cables from every GPU directly to every other (impossible with tens of thousands), you arrange switches in two layers and route data through the middle layer.

The bottom layer is the leaf switch — the switch GPUs plug straight into, each leaf handling one group of GPUs. The top layer is the spine switch — the switch that links every leaf together. When one GPU sends data to another GPU on a different leaf, the data always travels just two hops: leaf → spine → leaf, no matter how far apart those two GPUs are — which keeps latency "steady" and predictable, and that matters enormously during all-reduce.

What makes the switch chip "hard" is the word non-blocking — the network has to be built so every GPU can send at full speed to any other GPU at the same time, which means the spine layer has to bring down as much bandwidth as the leaves take in (a 1:1 ratio). The wider a single switch chip can send and receive (the higher its radix), the fewer switches it takes to build this fabric — meaning shorter cables, fewer switches, lower latency, and lower power bills. That's why "the width of a single chip" has become a key battlefield.

There's one more layer not to confuse: connecting GPUs within the same rack as fast as possible (called scale-up — Nvidia uses NVLink) is different from connecting across racks into a big cluster (called scale-out — using InfiniBand/Ethernet). The leaf/spine we just looked at is the scale-out layer — and it's the main arena of this lesson.

The switch layer doesn't work alone. It's the joint that ties several trends together.

• Tying GPUs / AI accelerators into one machine: this is the deepest relationship. However good a GPU is, it's worthless if it can't talk to its peers — so switch demand is tied directly to GPU sales. The more GPUs you buy, the more switches you have to buy with them
• Always paired with Optical Interconnect: the switch decides where data goes (the electrical layer), but when it has to travel farther than copper can handle, it gets converted to light (the optical layer) — these two layers have to get faster together, or one becomes the bottleneck. And the line between them is blurring with co-packaged optics, which puts the light right on the switch chip
• A component of AI servers/racks: many leaf switches get installed while assembling racks and clusters — it's a standard part you can't do without in any AI data-center build
• Depends on semiconductors: a switch ASIC is, in the end, a chip you make at an advanced foundry (TSMC) — top-end switch chips use the same generation of technology as GPUs
• Eats huge amounts of power: high-speed switches and SerDes draw a lot of power — that's exactly why high radix and co-packaged optics matter, because they cut the energy per bit

The biggest story in this field in 2025 is Ethernet overtaking InfiniBand to become the main scale-out network for AI for the first time. Just two years earlier, InfiniBand held about 80% of AI back-end switch sales, because it's stable and low-latency. But by Q4 2025 Ethernet had taken more than two-thirds of switch sales in AI clusters — Dell'Oro reported that Ethernet was "more than twice" the market size of InfiniBand for scale-out work that year.

Why did it flip? Because the big hyperscalers don't want to be locked to one vendor — Amazon, Microsoft, Meta, Oracle, and xAI all turned to Ethernet, which is open-standard, cheaper, and has multiple suppliers. The one who benefits the most is Broadcom, the king of switch chips, which in mid-2025 launched the Tomahawk 6 chip hitting 102.4 Tbps in a single chip (double the previous generation, holding 64 ports of 1.6T) and entered production in early 2026 — the world's first switch chip to reach this level.

ภาพประกอบ (battle.webp)

But Nvidia won't give up easily — it stopped selling just GPUs and started selling "the whole system," including its own networking: both InfiniBand (from the Mellanox acquisition) and a new Ethernet line called Spectrum-X designed specifically to fight Broadcom's Ethernet. In 2025, Spectrum-X alone crossed a run-rate of over $10 billion a year, and Nvidia's total networking revenue reached $8–11 billion a quarter — up more than 160% year over year. Meanwhile Arista, which focuses on selling high-end Ethernet "switch systems" to cloud titans, brought in $9.0 billion (+29%) for full-year 2025 and is targeting $2.75 billion in AI-specific revenue for 2026.

The first direction is the relentless chase for speed. Today's mainstream is the 800G port (which passed 20 million ports within just 3 years of shipping), and the next generation, 1.6T, is flowing into the market. Chips like Tomahawk 6 are already designed to support 1.6T directly. Every time the per-lane speed rises, copper runs out of road faster (the faster it goes, the hotter it gets and the shorter the distance it can carry), forcing the switch layer and the optical layer to fuse even more through co-packaged optics (CPO) — putting the laser right beside the switch chip. 2026 is seen as the first year CPO begins real production on both Ethernet and InfiniBand.

ภาพประกอบ (cpo.webp)

The second direction is open standards taking even more share. The Ethernet camp banded together as the Ultra Ethernet Consortium and released spec 1.0 in mid-2025 to reshape Ethernet into something low-latency and lossless enough to fight InfiniBand head-on for AI work. Dell'Oro expects AI back-end switch revenue to pass $100 billion a year by 2030, mostly Ethernet — but InfiniBand won't vanish. It still owns the jobs that demand the very lowest latency and highest certainty (frontier-grade training clusters).

The third direction is a new round of battle at the scale-up layer. Today, connecting GPUs within a single rack (scale-up) belongs to Nvidia alone via NVLink. But there are efforts to push Ethernet down into this layer too (such as an open scale-up standard). If they succeed, it opens the field for Broadcom, Marvell, and other switch-chip players to compete in a layer Nvidia used to monopolize — the layer with the highest value per port in the whole cluster.

The first risk is Nvidia's vertical lock. Because Nvidia controls both the GPU and the network, it can "sell as a package" and lock customers into buying its own switches along with the chips — forcing players who sell only switches (Broadcom, Arista) to fight against the pull keeping customers inside Nvidia's closed system. Even with Ethernet overtaking overall, breaking into clusters built purely on Nvidia GPUs gets harder.

The second risk is a standards war that isn't over. The InfiniBand vs Ethernet fight, the opening of scale-up, and the arrival of CPO all mean today's leader could be disrupted by a new standard or architecture. Betting on the wrong standard, in a business that changes fast and invests heavily in R&D, is very expensive.

The third risk is being tied to the AI investment cycle and a handful of customers. Switch demand is tied directly to building GPU clusters, and the big customers number just a few (the hyperscalers). If AI investment slows, or there's a stretch of "overbuilding," switch orders stall instantly — a business that grows extremely fast on the way up but is sensitive to the spending rhythm of a tiny set of customers.

In short: this node is the layer once seen as just "wires and boxes" that has turned into one of the most important bottlenecks and battlefields of the AI era — because the world's most expensive GPUs are instantly worthless without a switch wide and fast enough to tie them together. For moving data over long distances with light, read on in the sibling lesson Optical Interconnect & DCI.