The Return Of The CPU

To make sense of the variety, we sort chips by the job they do. Memory offers a useful ladder of speed, capacity, and cost. Processing chips are harder to arrange. Nvidia's Vera can sit beside GPUs or power separate machines that execute agents' instructions. Its networking chips contain substantial processors of their own. Even memory stacks include logic that manages data flow. The families overlap because the jobs overlap.

Software gives these shifting families some continuity.“GPU” remains a useful name even when the chip spends its day running AI rather than drawing images. CPUs retain an enormous base of ordinary software, even as their place in the machine changes. The familiar names survive; the division of work underneath them keeps moving.

That movement runs in both directions. A task leaves the CPU when specialized hardware can do it faster or use less power. The specialist takes on more jobs and gradually becomes a computer itself. Sometimes the CPU gains new capabilities and takes back work. Computer designers recognized this cycle in the 1960s and called it the“wheel of reincarnation.” Today, networking, storage, and security chips handle work that would otherwise occupy the server CPU, while CPUs are adding capabilities for more demanding AI calculations.

This helps reconcile the two counts in our introduction. Specialization creates openings for more chip designers. But the companies building whole systems want the pieces to work together, giving them a reason to design more components themselves and a powerful say over which outside suppliers get in. They don't need to make every chip to control those choices. More designers across the industry can therefore coexist with fewer suppliers inside a particular system.

That is the tension running through the families that follow. A growing job can create a market for independent suppliers, or become another component designed for a system owner's own use. In each family, we ask the same questions: what job does the chip do, who decides which chip fills it, and how many real contenders remain?

The first family is the CPU that keeps an accelerator fed. In a conventional AI server, two Intel or AMD processors might support eight GPUs, loading data and running the surrounding software. The CPU was usually a standard part, chosen from an established range.

Nvidia's integrated racks change that arrangement. Its CPUs connect to its GPUs through fast links that let the CPU's memory become additional working space for the GPU. The CPU is, in part, a memory warehouse. The rack designer also settles the processor choice before the customer orders. The balance has shifted: a Vera Rubin rack has one host CPU for every two GPUs.

Other processors inside the networking chips handle different jobs. A separate companion rack uses specialized chips to accelerate answer generation. Even around the GPU, the work keeps dividing.

The pattern extends beyond Nvidia. Google, AMD, and Huawei pair their accelerators with their own CPUs. But owning the system does not require making every chip. Intel still supplies CPUs for some Nvidia servers and is developing custom processors for Nvidia's systems. The company that once defined the standard part is also becoming a supplier of designs tailored to another company's platform.

The money follows the system. Nvidia gains another component to sell and greater control over how the machine performs. CPU speed matters, but buyers ultimately pay for the useful work the whole system delivers. Memory, connections, and software all help determine that result.

So in this family, the slot exists only inside someone's system, and the system owner normally fills it with its own CPUs. Competition happens one level up, between rival systems. A merchant CPU maker can win here only by being chosen by a system owner, or by selling to the buyers who still build their own.

The second family is the ordinary server processor in its new job: giving agents a place to live. Each agent needs its own small, sealed-off computer with files, memory, and tools, sometimes for hours. Cloud companies and AI labs now run these by the hundreds of thousands. This is the slot where the“CPU is back” story is most accurate.

The designs balance two priorities: fitting more agents onto each server and finishing each agent's work faster. Waiting agents need memory; busy agents need processing speed. Intel, AMD, Arm and Nvidia strike different balances between cores, memory and power consumption. Different jobs may produce different winners.

Standard processors remain strong here. Enterprises buy through server makers. They want their existing software to work and have little reason to design chips themselves. Compatibility protects the established market, where AMD has been gaining ground against Intel.

The cloud giants have the scale to make their own economics. Amazon, Microsoft, Google, and Alibaba all design server CPUs. Amazon's own chips already provide more than half the new CPU capacity it adds. Across such large fleets, even small savings can repay the design bill.

Meta spreads its bets across suppliers and also rents computing capacity from Amazon. Qualcomm is returning to servers with Meta signed up, and production planned for 2028. Its purchase of Modular, an AI software company, makes the point: a good chip also needs software that makes it easy to use.

Arm sits underneath most of the challengers. Its architecture and ready-made designs lower barriers to entry, giving many competing chips a common foundation. Some companies still design their own cores. Arm also sells finished processors, making it both supplier and rival. Its long-running licensing fight with Qualcomm, which reached another jury verdict this month, adds to the friction.

This family leaves room for established vendors, new challengers, and the cloud giants' own chips. The winner depends on the buyer's software, scale, and cost of getting work done.

The third family began as a humble network card. A decade ago, cloud companies noticed that their expensive server CPUs spent much of their time on chores: moving network traffic, handling storage, encrypting data and keeping one customer walled off from another. So they moved the chores onto dedicated chips. Amazon led. Google, Microsoft, Nvidia and AMD followed.

These chips have since grown into computers. Nvidia's latest carries nearly as many processor cores as a mid-sized server CPU. Intel, the company that defined the standard server, sells one built on Arm cores. The wheel of reincarnation is turning in plain sight. The work left the CPU, and another computer grew around it.

Agents give these chips a new purpose: acting as an independent guard. In July, agents in an OpenAI test environment crossed their intended boundaries and attacked another company's systems. A guard running alongside the agent can be vulnerable itself. Nvidia's latest security design puts an additional layer on the network chip, separate from the software running the agent, to check and restrict its access.

These chips also handle storage tasks, including retrieving information that AI models reuse between calls. The boundary between networking, security and memory keeps blurring.

Cloud giants build their own versions. Nvidia integrates its chips into its systems. Independent suppliers compete for other buyers. The commercial attraction goes beyond saving CPU time: once a chip becomes part of a customer's security and operations, replacing it becomes a much bigger decision than changing a network card.

The fourth family is the least visible and, we think, the most misunderstood. It is the logic that decides where data sits and how it moves.

AI models keep a running record of each conversation. They need not reread everything for every new word. Agents make that record balloon. They hold long tasks, revisit the same documents, and keep many sessions open at once. The fastest memory is also the scarcest and most expensive. For two decades, computing power has grown far faster than the ability to feed it. Memory has become a ladder, from memory stacked beside the GPU down to ordinary server memory, pooled memory, and flash. At each rung, a chip now decides what goes where.

As we argued in“Memory by Design”, AI inference is becoming a memory system as much as a compute system. Agents that barely touch their processors still keep their memory occupied.

Two consequences are easy to miss. Some of today's“CPU demand” is really demand for the memory a CPU can host. Better memory chips can reduce CPU demand. A buyer who adds a processor only to get more memory may not need it once memory can be added on its own.

Control is split here. Where an industry standard is mandatory, a few merchant suppliers share the market. Where memory is built for one customer, design goes in-house. Investors who watch only memory prices and GPU counts miss the logic in between.

Outside the data center, the same splitting happens inside a single chip. A flagship phone processor already holds a CPU, a GPU, an AI engine, an image processor, a modem and a security vault. On-device agents are reshuffling the jobs among them. The AI engine runs the model. The CPU runs the agent's loop of planning and tool use. As in the data center, memory is often the real limit.

The same core designs now cross categories. One Arm core sits in a MediaTek phone chip and in Nvidia's PC and desktop AI chips. Qualcomm uses its cores in phones and PCs, and plans a version for servers.

In-house design spreads fastest where one company owns the whole product. Apple designs the main processors of its phones and Macs. Samsung and Huawei design many of theirs. Phone makers now design about a third of all phone processors. Tesla, BYD, NIO, Xpeng, and Li Auto design their own driving chips. Merchant suppliers stay strong where buyers are many, in most PCs and most mid-range phones.

On devices, the division of work happens inside one package, and the company that owns the device increasingly designs it.

China shows the division of work at its most extreme because it has had little choice. Export controls deny it the best GPUs, the latest memory, and the most advanced manufacturing tools. Its answer is to use more chips, each doing a narrower job, and to link them tightly.

Huawei's CloudMatrix is what that looks like. It ties together five times as many accelerators as Nvidia's flagship rack, plus hundreds of CPUs and switch chips, spread across 16 racks and linked by thousands of optical connections. It runs three separate networks, each with its own chips. The cost is power: by most estimates, it draws close to four times as much as Nvidia's rack for well under twice the computing.