AI Industry Structure, Layer 2: GPU & Chip Design (Fabless)

If you had to pick the single highest-value-added layer in the entire AI industry structure, this is it: the fabless layer that designs GPUs. If you want to know why NVIDIA became one of the highest market-cap companies in the world, this layer holds the answer. In this article, we’ll cover exactly what a fabless company does, why GPUs are so well-suited to AI computation, what other AI chips exist besides GPUs, and why design and manufacturing are split apart.

Revisiting Fabless

This layer is where the most expensive and important component in the entire AI industry structure gets made — the AI compute chip, the GPU. NVIDIA is the leading company designing these AI GPUs. But why did the GPU, of all things, become the star of AI computation instead of the CPU?

Why GPUs, of All Things?

A CPU is specialized for processing one complex task quickly, in sequence. A GPU, on the other hand, is specialized for processing thousands or tens of thousands of simple calculations simultaneously. It was originally built to draw millions of pixels on a game screen all at once. As it happens, AI computation (repeated matrix multiplication) is the exact same kind of task — doing a massive number of simple calculations at the same time. That’s why a GPU, developed for gaming, turned out to be a perfect fit for the AI era.

NVIDIA was the first company to notice this near-accidental fit and redefine the GPU for AI computation. Starting in the mid-2000s, it steadily worked to make GPUs usable for general-purpose computation, not just gaming. That work culminated in CUDA’s official launch in 2007 — timing that lined up perfectly with today’s AI boom.

Are There Other AI Chips Besides GPUs?

Strictly speaking, there aren’t many companies making true “GPUs” — really just NVIDIA and AMD. But broaden the lens to “chips for AI computation” as a whole, and the picture changes. Everyone is trying to route around NVIDIA in ways other than the GPU.

The most common approach is the ASIC (application-specific integrated circuit). Google’s TPU and Amazon’s Trainium are the classic examples. Unlike general-purpose GPUs, these are dedicated chips optimized only for their own company’s workloads. Interestingly, chips like Google’s TPU are often co-designed with Broadcom rather than built entirely in-house from scratch. Broadcom’s presence has grown significantly lately. It now acts as a kind of “agency” that designs custom AI chips on behalf of big tech, to their exact specifications.

Intel is also making its play with a separate AI-dedicated chip called Gaudi instead of a GPU. Meanwhile, startups like Cerebras, Groq, and SambaNova have come out with chips designed on architectures completely different from the GPU. Cerebras, for example, makes a massive chip that uses an entire wafer as a single chip. Groq, meanwhile, builds chips specialized for inference speed (the step where a model answers a question).

NVIDIA’s Real Moat Isn’t the Chip

But even with all these competitors, there’s a reason NVIDIA’s position isn’t easy to unseat. Here’s a common misconception: NVIDIA’s edge isn’t simply that “its chip is fast.” Its real weapon is the CUDA software ecosystem. CUDA is a toolkit that lets developers write programs that harness the GPU. Most AI researchers and developers worldwide are already fluent in it.

So even if a competitor matches NVIDIA “on chip performance alone,” developers won’t easily switch. Not if it means relearning a whole new toolkit from scratch. This is what really makes NVIDIA strong. It’s not the chip itself, but the “lock-in” created by the software and developer community built around it.

What Exactly Does CUDA Do?

GPUs are naturally great at handling “a massive number of simple calculations at once.” But someone still has to specify exactly how to split that computation across thousands of GPU cores. CUDA is a programming tool that lets developers give that instruction using relatively simple code. In effect, it lets them say: “split this calculation across the GPU cores like this.” Before CUDA existed, using a GPU for general-purpose computation meant awkwardly repurposing graphics-only instructions. That made the barrier to entry far higher.

NVIDIA has been steadily refining CUDA for nearly 20 years now, since 2007. Over that time, nearly every major framework AI researchers use (like PyTorch and TensorFlow) has been built around CUDA. Everything from university courses to online tutorials has piled up around it too. So no matter how good an alternative a competitor offers, switching isn’t easy. It’s hard to get “the massive body of code already written in CUDA, and people’s habits” to move all at once.

Design and Manufacturing Are Fully Split

Chips don’t just fall from the sky. Deciding how to lay out tens of billions of microscopic circuits onto a chip the size of a fingernail is an extraordinarily precise design job. That has to happen first. But NVIDIA doesn’t actually own a factory that stamps out chips. It only designs them and hands production off to someone else.

There’s a reason it doesn’t own a factory. Building a single state-of-the-art foundry (manufacturing plant) costs tens of billions of dollars. Keeping that technology at the cutting edge also requires massive reinvestment every year. For NVIDIA, pouring that money into design and R&D instead is a far better deal. So it hands production off entirely to specialized foundries like TSMC.

What Investors Should Watch For

The question investors ask most often about this layer is “how long will NVIDIA’s dominance last?” Judging that requires looking at these factors.

It’s easy to miss this if you only watch the chip performance race. But the real contest in this layer comes down to “how well a company has bundled hardware and software together as a whole.”

Closing Thoughts

GPU and chip design is the highest-value-added layer in the AI industry structure. But the blueprint alone can’t produce anything on its own. Someone still has to actually stamp that design into a physical chip.

In the next article, we’ll take a close look at that final piece: foundries. We’ll cover why TSMC alone effectively holds a monopoly on leading-edge chip production worldwide.