The chips that 'think for us' — and the one company holding the tap of the AI era

The first reason is the sheer scale of the money. In fiscal 2026 NVIDIA's revenue hit $215.9 billion, up 65%. Its data-center chip business alone topped $200 billion — up from $115.2 billion the year before. This isn't ordinary company growth; it's one company pulling in the world's AI investment as a single main stream.

The second reason is that this market is ballooning. The AI accelerator chip market was around $140 billion in 2024, and many analysts expect it to grow past $400–600 billion by 2030–2033, at roughly 16–25% a year. Some (Bloomberg Intelligence) even see accelerators alone topping $600 billion by 2033 — and counting the whole stack (memory, networking, CPU) it could reach ~$1.7 trillion.

The third reason is that 'shortage' is no exaggeration. NVIDIA has booked out nearly all of its new Blackwell (B200) chip capacity through mid-2026, allocating it first to big customers like Microsoft, Google, Amazon, and Meta. The bottleneck isn't demand — it's production capacity, especially the advanced packaging step (CoWoS) that bonds HBM to the GPU.

To understand everything in this lesson, you need one fact: almost all of AI, underneath, is 'matrix multiplication' — taking two big tables of numbers and multiplying them, over and over, billions of times. A large language model with trillions of parameters is, at its core, a giant pile of matrices being multiplied without rest.

Here's where the magic is: matrix multiplication 'breaks into small pieces that can be computed at the same time' — each cell of the result can be computed separately, without waiting on the others. That's why a CPU, with few cores that do one thing fast, can't keep up — while a GPU, with thousands to tens of thousands of tiny cores, can split the multiplication into small pieces and charge through them all at once.

But this opens a second option too: if the job really is just matrix multiplication, why not design a chip that does only that job, as well as possible? This is the idea of the ASIC — a custom-built chip. A GPU is good at many things (flexibility is its strength), but an ASIC like Google's TPU strips out everything unnecessary and leaves only a giant matrix-multiply unit. The result: on the job it was designed for, it's usually cheaper and more power-efficient per computation — at the cost of not doing anything else, and being harder to develop.

AI chips work in two very different modes, and the difference is shaping the future of the whole industry.

• Training: teaching a model once, on a huge scale — it eats enormous power and needs tens of thousands of GPUs wired together for months. It's complex, flexible, and very tightly bound to NVIDIA's software
• Inference (running it): taking the trained model and 'running' it to answer real questions — happening millions of times a day, every time someone types a query. It's repetitive and predictable, so it suits cheaper custom chips

The turning point is this: as AI spreads, real-world use (inference) keeps outweighing training. And this is the weakest point of NVIDIA's moat, because the moat — called CUDA — was built mainly for 'training.' As the world tilts toward inference, the need to lean on CUDA loosens — opening a gap for the tech giants' ASICs and inference-specific chips (Inference-Optimized Silicon) to slip in.

The economic logic is dead simple: if Google's, Amazon's, or Microsoft's custom chips can do inference at about 80% of a GPU's performance but at roughly 40% lower total cost (TCO), these giants have a huge incentive to move their own internal inference off NVIDIA — and that's exactly what's happening.

The AI chip layer doesn't float alone — it's the point where several big trends meet:

• The foundation of all of AI: models, apps, and AI agents — everything in the layers above sits on top of this chip layer. If chips run short, the whole trend stalls
• Meets Semiconductors across the board: AI chips are the chip industry's biggest customer, spanning compute (Logic), memory (Memory / HBM), and manufacturing (Foundry) — TSMC makes nearly all the most advanced AI chips
• Draws power from Energy Transition & Power: AI data centers eat so much electricity they've become a major driver of global power demand — this node 'drives demand' for energy directly
• Depends on Critical Materials: high-quality silicon wafers, specialty chemicals, and rare materials are indispensable raw inputs
• Starts with design software (EDA & IP): before any chip exists, every design passes through the design software (EDA) of Synopsys/Cadence and IP blocks like Arm's cores — the upstream layer that collects a toll from every camp (see EDA & Semiconductor IP)
• Not just in the data center: not every AI chip runs in a data center — the NPUs in phones, PCs, and cars run inference on-device for speed and privacy (see Edge & On-device AI Silicon)
• Unlocks other trends: AI chips 'enable' Robotics, Biotech, and Cybersecurity — they're the infrastructure other trends stand on

From a value-chain view, this node is the 'bottleneck' where power is most concentrated, because it bundles three hard layers together — chip design (where NVIDIA leads), manufacturing (where TSMC monopolizes the cutting edge), and HBM memory (made by just 3 companies). Anyone who wants to compete has to break through all three gates at once.

The picture today is 'one giant + an army of challengers.' NVIDIA still held about 80–90% of the AI accelerator market by revenue in 2024–2025 (it touched ~90% in 2024). But analysts expect its share to slip gradually to ~75% by 2026 — not because NVIDIA is growing slower, but because the whole pie is growing faster than any one company can eat it.

The difference that tells the story most clearly is margins. In the quarter ending October 2025, NVIDIA's gross margin was about 73%, while AMD — the #2 rival — was around 50%. That gap is the 'CUDA moat toll' NVIDIA collects — and the reason every tech giant wants to design its own chip to dodge it.

The custom-chip side's pursuit is gaining real weight: Broadcom and Marvell handle almost all the ASIC design for the tech giants, together holding about 80–95% of this market, and custom-chip orders are surging (who's ordering how much from whom is in that node's own lesson).

The first direction is the shift toward inference. As AI gets embedded in every app and device, 'running' models will keep eating more compute than 'training' them — a battlefield where custom and specialized chips have the cost advantage. This is the gap that will erode NVIDIA's share (but not its leadership) bit by bit.

The second direction is every tech giant will have its own chip. Google, Amazon, Microsoft, Meta, and OpenAI are all pouring money into ASICs through Broadcom/Marvell to reduce their NVIDIA dependence and control their own cloud costs — shifting the landscape from 'everyone buys from NVIDIA' to 'your biggest customers become your competitors too.'

The third direction is AI chips moving to the edge. Not every AI job needs to run in a data center — phones, cars, and IoT devices are starting to carry AI chips of their own (Edge & On-device AI Silicon) for speed, privacy, and cloud-cost savings — opening a new market where mobile players like Qualcomm and Apple have the edge.

The fourth direction is the bottleneck moving. Today the bottleneck is HBM and packaging (CoWoS); tomorrow it may move to power and cooling — because AI data centers eat so much electricity they're starting to hit the limits of the power grid. Whoever solves the next bottleneck first gains the advantage.

The appeal of the AI chip layer comes with risks all its own.

The first risk is extreme concentration. Enormous value is tied to just a few companies — NVIDIA on design, TSMC on manufacturing, and just 3 HBM makers. If any one point stumbles (disaster, geopolitics, or technical), the whole chain shakes. And because the most advanced manufacturing is concentrated in Taiwan, geopolitical risk is far from remote.

The second risk is the CUDA moat may not be as wide as it was. As long as the job is training, CUDA stays sticky — but as the world shifts to inference, the dependence on CUDA loosens. If the tech giants' custom chips work well enough at a much lower cost, the leader's 73% margins and its share could be squeezed — the chip industry's lesson is that 'today's leader can fall by missing a single technology wave.'

The third risk is the cycle and 'bubble' question. AI chip demand is so strong right now because the tech giants are pouring in enormous capex all at once. If AI's payoff comes slower than hoped, or investment slows, the demand that looks bottomless could shrink fast — because most of it comes from the same few deep-pocketed customers.

In short: AI accelerators are the 'tap' that can genuinely turn the AI era on and off. Today one company holds that tap, more through software than the chip itself. But the field is shifting — as the world moves from 'training' to 'real-world use' and the biggest customers start building their own chips. Understand this layer and you understand why it's both the heart and the bottleneck of the entire AI economy.