The Chip Industry's Future Is Memory By Design

(MENAFN- Asia Times) In 1884, an English schoolmaster wrote a short novel about a world with only length and breadth, whose inhabitants could not conceive of a third direction until one of them was lifted out of the plane and shown his own country from above.

Semiconductors have lived in that world for 70 years. Transistors were drawn on a flat surface, wires were run across it, and progress was measured by how small a distance could be maintained between two features.

The single number, quoted in microns and then nanometers, became the industry's scoreboard, its marketing, and eventually its mythology, for the simple reason that in a flat country there is only one direction in which to get better.

The most consequential decision in computing was itself a piece of flat-world planning. When the stored-program machine was described in the 1940s, computing and memory were placed in separate boxes with a channel between them (Von Neumann or Princeton architecture ), and the machine has spent its life since then fetching from one to feed the other.

It was the natural arrangement, and for a long time the right one. On a plane, two things cannot occupy the same place, so anything that must be large and anything that must be fast are built side by side and connected.

Seven decades of engineering then went into widening that channel and shortening it, which is what caches, wider buses, and eventually stacked memory placed beside the processor have all been doing.

Things started going vertical, finally, in the 2010s. The third direction arrived first in memory, which is the part of the story people forget. Flash went vertical more than a decade ago because its cells had stopped shrinking usefully, and stacked DRAM followed.

Even then, the serious work stayed flat. The stack was a taller building in a two-dimensional town, joined to the processor next door by a road. It was appropriately given the tag of 2.5D for its two-and-a-half-dimensional nature.

That is what is now changing, and quickly. Partly because demand is enormous and impatient. Partly because more money is available for this research than at any point in the industry's history.

And partly because fields that used to advance separately - memory process technology, logic foundry technology, packaging, thermal engineering, and design software - have all arrived at the same problem in the same year.

Our strategic industry consultant, Vibhu Sharma, is the co-author of this piece and has prepared thorough visual exhibits for this note. Each is rich in detail and worth a write-up of its own.

These diagrams illuminate intricate architectural nuances that might otherwise remain opaque. Readers who have difficulty accessing or viewing the exhibits are encouraged to reach out to our team.

A giant aside: the week the fluid blew up

This may not be the E equals MC squared of our era, but the week the Navier-Stokes problem falls, or claims to have fallen, to a machine is not a week to open a note with anything else.

Ten thousand agents, 88 hours, 130 billion tokens, and a question posed precisely in 1934 and unanswered ever since is resolved, or at least such is the claim, in the direction nobody was rooting for: the equation break. The mathematical community has not yet ratified it, and the prize has not been claimed, but even the claim is a big deal.

The horde claiming that the machines are saturating, or that AI is nothing but a rearrangement or a statistical parrot, is thinning fast. Instead, GenAI's surpassing of the limitations of skull and skill is producing extraordinary results at a breathtaking clip now. There is another important claim here, and it stems from what led to the proof's generation.

Apparently, the folks at OpenAI heard a rumor that two mathematicians, one of them an Anthropic researcher, were close to a Millennium Problem. It pointed its machines at the problem on the first of September and had a Lean-verified proof by the sixth. The rumor turned out to concern a related but different problem.

The anecdote has an important lesson, even if stated overly simplistically: a party thought a proof of the Millennium Problem was possible, started working, and found solutions to one of the most difficult, unsolved problems in days.

We're bringing back one of our favorite concepts we haven't talked about much lately: instant copyability. There are amazing innovations where leaders have few moats, and almost all of them are in the application and cognition layers. For successful, money-making innovation, the scarce thing is no longer the discovery. It is a discovery that cannot be immediately copied or improved upon by someone else.

That is the bridge to everything below. Innovation is exploding, and not only in mathematics. Anyone who attended the presentations at SEMICON in Taiwan will have come away with the same impression: innovation is exploding in multiple directions. When the frontier moves this fast in this many places at once, the useful question stops being who is ahead and becomes where the moats are, or where they might still be built.

The rest of this letter is about one such place: the boundary between memory and logic, which is being redrawn now, and where the answer is not yet obvious.

A theme that does not yet have a name

The conversation of the past two years has been about memory hierarchy. Shortages of high-bandwidth memory amid exploding demand spurred an urgent search for tiers: more capacity somewhere in the middle, flash-based structures reaching for bandwidth, and new pools sitting between the expensive, fast layer and the cheap, slow one. There has been a slew of innovations and ambitions along the axis.

Meanwhile, something else has been happening without a name attached. Memory and logic have begun to merge. In our frame, the memoryland and logicland that we so elegantly contrasted to describe our own odyssey weeks ago could not have been better timed, as now the two have begun to merge, and we are likely headed to an era where it may not be possible to call a company just a memory or a logic foundry.

Samsung is pushing memory downward into the processor's territory and, ultimately, on top of it. Nvidia is pulling the memory controller and the interface upward into its own platform, and selling that platform to everyone building custom silicon. TSMC has quietly become a supplier of memory foundations, because those foundations are now logic.

SK Hynix is building the same capability through partnerships, not ownership, and by putting advanced packaging close to its customers. Four large companies are moving in four different directions across the same square of ground, and there is no umbrella term for what they are collectively doing.

Underneath all four movements sits a single fact with commercial consequences, and it is the fact to carry through the rest of this note. Memory is no longer a pluggable part. For 30 years, its defining property was that any qualified supplier's product could be dropped into any customer's socket, and every move described below takes some of that property away.

When the most supply-constrained component in the machine is no longer interchangeable, everyone who touches it has to move, including those who were perfectly happy with the old arrangement.

In late 2025, we announced custom chips as one of the emerging dominant themes to occupy our headspace in 2026. We think memory and logic contamination into each other's territory, with whatever term they go by later, is likely to emerge as a dominant industry theme from 2027.

Three directions at once

The technical story is easiest to hold if it is separated into the three directions in which the memory frontier is expanding, because for the first time all three are moving together. Exhibit 1 puts the whole progression on one line, generation by generation, and two things are worth noticing about it.

The mechanism of improvement changes as the eye travels right. And every arrangement up to the second-last one is the same arrangement: processor beside memory, both sitting on an interposer, which is the flat town with its road. Only the panel at the far end looks different.

The first direction is width and speed. For several generations, more bandwidth came from running each connection faster across an interface of essentially fixed width: the same 1,024 connections, driven from 3.6 to 5.6 and then to 8 gigabits per second. That approach became expensive in power and design margin, so the current generation doubled the number of connections instead.

The table shows the moment the lever changes, and it is worth pausing on the column that names the mechanism. When the answer stops being“faster” and becomes“wider,” the interface stops being a standard and becomes a design decision.

Latest stories Bessent's big talk runs into bigger market realities US calls for AI poisoning to sabotage China's model distillation Trump laughing all the way to the bank... at the end of the world

The second direction is height. Capacity grows by stacking more memory dies on the same footprint, and the industry is doing this within a package height that refuses to move. This means every additional layer is paid for in thinner dies and tighter bonds.

The exhibit below shows the two directions as they are usually drawn: bandwidth across the plane and capacity up the stack. The useful thing about seeing them together is that neither is sufficient alone. A wider interface to a small pool of memory simply reaches that pool faster. A deeper pool behind a narrow interface starves.

The third direction is the interesting one, and it is architectural rather than physical. It concerns what the foundation beneath the memory stack is allowed to do.

Logic beneath the memory

A memory stack sits on a base die. For most of the product's life, this was a plain floor: wiring, a physical interface, some test circuitry, nothing anyone would call design. Its size was never really its own decision, because a foundation must be as large as the building above it, and the building is a stack of memory whose dimensions are set by capacity.

Then the floor moved to an advanced logic process. The current generation pairs leading memory dies with a foundation built on a four-nanometer logic node; the half-step after it stays there, and the generation beyond is already slated for two nanometers. Exhibit 4 lays out that pairing and the roadmap behind it.

Note what it implies: a slab of expensive, capable logic silicon now arrives underneath every memory stack, sized by the memory above it, whether anyone uses it or not.

Leaving that slab as routing is like buying land in the middle of a city to park a bicycle, and all three memory makers have now said what they intend to build there.

The first tenants are borrowed from the processor. The memory controller moves down into the foundation, and the bulky standard interface gives way to a shorter, narrower link, because two dies bolted together do not need the ceremony an open standard demands. Both moves hand expensive silicon back to the processor and shorten the distance data travels.

Samsung's roadmap, presented at Hot Chips in August, then keeps going: repair that fixes individual cells instead of swapping out whole rows, sensors reporting voltage, aging and manufacturing variation, an outward-facing port turning the foundation into a gateway to cheaper memory beyond the stack, and eventually small processing elements doing simple arithmetic where the data already sits rather than shipping it to the processor and back.

At SEMICON a fortnight later, the company gave the strategy its label, CUBE, for capacity, utilization, bandwidth and efficiency, and bet the roadmap on three-dimensional architectures.

The others are on the same road with different vehicles. On the same Taipei stage, SK hynix argued that memory makers must become co-designers of memory-centric AI systems, which, if it happens, will make Hynix encroach on the territories currently occupied by the likes of Nvidia and Broadcom.

It has since proposed applying logic foundry techniques to DRAM's peripheral circuits, with one of its executives observing that this is dissolving the boundary between the fab and the packaging house. Micron is the plainest of the three about why any of this is worth doing: it will offer its next generation in both standard and customized versions, and says the customized version should carry higher gross margins than the standard one.

That single sentence is the whole thesis in commercial form. Each step is defensible alone. Together they turn a component into a subsystem. Everyone seems to be learning from Nvidia's playbook.

The tell is in who builds the foundations. Samsung makes its own, on its own four-nanometer logic. SK Hynix and Micron both buy theirs from TSMC, or, in the future, either from another foundry or by developing the capabilities themselves.

A memory company that specifies its own controller and its own PHY has taken a small step into the design business. A logic foundry that supplies the floor has taken a large step into the memory business. Neither of them announced it that way, though.

Both sides want the same floor

Here is the awkward part, and it is the reason this becomes a contest instead of a roadmap. Almost every function on that list is done today at the logic end, by the processor, and has been for as long as processors have had memory attached.

The controller belongs to the processor. So does the decision about how memory is addressed, when it is refreshed, how failures are handled, and how data is scheduled. Memory has historically been told what to do.

So when Samsung proposes moving those functions into the foundation, it is asking the processor to hand over territory it currently holds. And the processor companies have noticed. In August, Nvidia disclosed an arrangement placing its own memory controller inside the base die, with several memory suppliers building to one specification that Nvidia itself defines. The engineering direction is identical. The ownership is opposite.

That is worth sitting with. Both parties agree publicly on where the industry is going. They agree that the foundation beneath the memory stack should hold the controller, that the standard interface should be replaced with a shorter, bespoke one, and that this frees processor area and saves power. What they disagree about is who designs it, and therefore who is paid for designing it.

This is not seen as a dispute, but as we move into 2027, some of these friction points will become more obvious.

Designers move first

As soon as one raises the question of“memory by design,” the title of this piece, the obvious question is who designs all that new silicon. The answer would not surprise anyone.

In August, Nvidia disclosed an arrangement placing its own memory controller inside the base die, with several memory suppliers building to one specification that Nvidia itself defines.

The claimed results are exactly what the free-real-estate argument predicts: substantially more bandwidth, meaningfully less memory power, an interface footprint cut by as much as two-thirds, and up to a quarter of the processor's own area handed back for computing.

The exhibit below shows the before-and-after, along with the arithmetic for what gets reclaimed. On the engineering, there is little to argue with.

On the economics, notice what has happened. The design conversation that used to take place between a memory vendor and its customer, and which was the source of whatever switching cost the memory vendor enjoyed, has moved into the platform layer and been standardized.

Three suppliers building the same thing to somebody else's drawing is a description of a commodity, however sophisticated the drawing. The final panel of that exhibit is the one to read twice: the relationship it describes as tomorrow's, in which processor architecture, memory foundation and memory dies are co-designed together, does not specify who holds the pen.

Days later, the pattern repeated in a form that should worry memory more than the first. MediaTek adopted Nvidia's rack-scale platform as the foundation on which it will design custom accelerators for others.

The bundle includes connectivity, packaging, rack architecture, and, listed without ceremony, the memory architecture. Amazon's Annapurna Labs is the first named NVHBM partner, beginning with Trainium4.

The most self-sufficient custom-silicon team in the industry is taking its memory architecture from the platform.

Keeping the part pluggable

There is a second reason logic moved, and it has nothing to do with bandwidth.

For 30 years, memory's defining commercial property was interchangeability. A standards body published the drawing; several manufacturers built to it; qualification was close to a formality; and a processor company could hold three suppliers against one another every quarter. That property is why memory was valued as a commodity, and it usually deserved to be. It is also why memory was safe to depend on.

Everything in the preceding sections removes that property. A controller tuned to one architecture, a private interface, repair schemes matched to one fab's defect distribution, a foundation built on a logic node the memory maker chose: none of it leaves the part droppable into somebody else's socket. Micron's line about the customized version carrying a better gross margin is that same sentence read from the seller's side of the table.

Now consider it from the buyer's side: at the end of two years, the single hardest thing to obtain in the industry has been high-bandwidth memory. A processor company that allows its memory to become architecturally unique to one supplier has handed that supplier a veto over its entire product line, in the component it can least afford to lose. That is an unacceptable position for any customer and a spectacular one for any vendor.

Read that way, Nvidia's specification is as much an insurance policy as an architecture. Writing the drawing yourself is how a customer collects the benefits of co-design while keeping three vendors building the same object. The bandwidth and the power savings are real. So is the quieter function: to stop memory from getting away.

This is the tension that will run through the next several years, and it has three parties in it. Memory is climbing toward design, because design is where the control is. Logic is descending into memory, partly for the engineering and partly to keep its suppliers substitutable.

And the logic foundries, having been invited in to build the floor, now sit between them, holding a view of both roadmaps and an opinion about each. Nobody in this triangle is in the business they were in five years ago.

Memory refuses the boundary

Memory's answer, and it is still developing, is more radical than defending the foundation. It is attempting to remove the thing being fought over.

Conventional stacked memory is lateral. The processor sits on one side, the memory beside it, and they communicate across an interposer, which is a small, dense, expensive road. Every argument about who owns the memory controller is an argument about the traffic rules on that road.

Samsung's proposal, presented publicly more than once this year, is to lift the memory on top of the processor and remove the road. Memory above, computing below, bonded into a single object, with connections distributed through the structure instead of concentrated at an edge.

It is the panel at the far right of Exhibit 1, the only one in that row without an interposer under it, marked as an early roadmap item, which is roughly its current maturity and not at all its significance

If that sounds like an engineering preference, consider it as strategy. An existing boundary can be set by whoever has the most leverage. A boundary that has been abolished cannot be specified by anyone, because there is nothing left to write down. Memory's escape from commoditization lies in making itself inseparable from the thing it serves. It is the logical endpoint of ceasing to be a pluggable part: a component that cannot be unplugged at all.

The engineering price is severe and should be stated honestly. Bonding must become far finer than today's connections allow. Heat becomes harder to manage because shrinking the interface while raising bandwidth concentrates power in a smaller area rather than spreading it out.

That is why the same companies are now designing heat paths as carefully as data paths. Two dies must be designed and verified as one, which means memory people are supplying models and test structures into a design flow that was never built to receive them. And when something fails after bonding, the loss is no longer a memory stack. It is a memory stack fused to an expensive processor.

Why it is worth the trouble

The reason the industry will pay that price fits in one sentence: memory power saved becomes part of the compute power budget.

Modern accelerators hit a power ceiling long before they run out of transistors. A rack has a power envelope, a cooling system has a capacity, and every joule spent moving data between memory and processor is a joule unavailable for computing.

On a processor drawing more than a kilowatt, removing most of the interface circuitry frees on the order of a hundred watts, which converts into several percent more computing headroom while also delivering far more bandwidth.

This is also the answer to anyone who suspects the industry of over-engineering for its own amusement. The flat world has run out of easy directions. Processors cannot grow much larger, since there are physical limits to how big a die can be printed. Interposers cannot grow indefinitely either.

Signaling faster costs power that is no longer available. When every direction on the plane is blocked, one goes up.

The other boundary

The line between memory and logic is the one everyone is discussing. The second line is more interesting.

For four decades, logic has been organized around a separation between those who design and those who manufacture. It was the industry's great structural innovation; it made fabless design possible and produced the largest manufacturing company in the sector.

Memory never had that separation. Memory was manufacturing with a modest amount of design attached, which is precisely why it was valued as a commodity.

Both lines are dissolving at once, and in the same place. Exhibit 6 puts it plainly: two roadmaps that used to advance independently, a memory roadmap and a foundry roadmap, now have to advance together because a single product depends on both.

The consequences run further than the picture shows. When the foundation beneath a memory stack is a logic die on a leading node, the memory company buys advanced logic wafers that reportedly cost several times as much as the memory dies above them, and its foundry choice becomes one of its largest input decisions.

The foundry, having supplied that foundation, acquires a view of the memory architecture and a growing say in it. And the memory manufacturer, having spent decades learning exactly how its own dies fail, is in some respects better placed than anyone to decide where the interface should sit and what belongs in the foundation, in the same way that processor designers have always decided where to place fast memory beside their computing cores.

Design encroachment therefore runs in every direction at once. Logic firms are moving into memory architecture. Manufacturers are moving into design. Foundries are moving into both. The tidy taxonomy of the last forty years, in which one party drew the chip, and another built it, and memory did neither in any interesting sense, will not survive this decade intact.

Which is why Samsung's position deserves attention, whatever one concludes about its recent execution.

The ceiling is not the outcome. Vertical integration has disappointed before, repeatedly, including there. But the ceiling is real, and for the first time in years, the company's unusual shape is an advantage rather than an overhead.

Sign up for one of our free newsletters

The Daily Report Start your day right with Asia Times' top stories AT Weekly Report A weekly roundup of Asia Times' most-read stories

What cannot be specified

If both sides are advancing, the question is which advance holds, and the answer lies in what can be written into a document.

A controller can be specified. An interface can be specified. That is exactly why logic's move works: publish the drawing, qualify three suppliers, allocate volume, and watch switching costs fall away.

What memory is retreating toward cannot be written down the same way. How much defect to screen out before bonding, where repair resources should sit, what the bonding recipe should be, how yield learning accumulates: these are functions of a particular fab's own defect distribution and process history.

Two suppliers cannot be instructed to implement them identically, because they are not the same underneath. The switching cost that logic dissolves at the foundation reappears one layer up, in knowledge held rather than published.

As is widely discussed across the industry, the importance of packaging and testing has exploded, along the lines of memory last year. Many other changes are coming as we usher in the 3D era of semiconductor manufacturing.

Everything at once

Now, the most important part.

Historically, these advances arrived in sequence. A generation was designed, manufactured, and matured; only afterward did the industry attempt something structurally different. That courtesy has been withdrawn. The next exhibit is the whole argument in one picture, and we would put it on the wall. Twelve-high stacks sampled in March 2025 and production-ready by September.

Commercial shipments in February 2026, with samples of the next half-step three months later. The controller moves into the foundation in August. And in that same August, the next-next generation was presented publicly alongside the architecture that abolishes the interposer entirely. What used to take a decade across generations is happening in quarters, concurrently.

For investors, this has two consequences worth holding onto. First, production maturity and frontier position have come apart. A company shipping today's generation in volume may already be behind on the architecture that decides the next one, and both facts will appear in the same quarterly results without comment.

Second, when each new curve begins before the last has flattened, the durable advantage belongs to whoever carries learning forward faster than the frontier moves. That favors accumulated capability, qualification history, and co-design experience over any particular product victory.

The arms dealers, and the question that was retired

We will resist predicting who wins the boundary, since the honest position is that memory's most ambitious plans remain roadmap and concept while logic's are shipping. Two things can be said with more confidence.

Whichever way the boundary settles, a set of suppliers is paid to settle it: bonding equipment, testing, inspection, and those upstream. Wars are good for arms dealers, and this one requires a great deal of new equipment regardless of the victor.

And the old question has quietly been retired. For 30 years, the memory industry was judged by who could produce the most bits at the best yield, which is why it was valued as a cyclical commodity business. The question now is who can deliver the best combined memory and computing system.

That is a different business, with different economics and, we suspect, different multiples. The industry left Flatland some time ago. The way it is valued, it seems the departure has gone unnoticed.

Memory by design

We chose the title with three meanings in mind. Memory is designed, in the sense that it has architecture and authorship where it used to have a specification and a price.

Memory is designed by somebody, and the fight over which somebody is the subject of everything above. And the machine is increasingly designed around its memory, which inverts seventy years of practice in which memory was the thing accommodated last.

Memory and logic fusion will be a major theme of the coming years, and it arrives alongside several others rather than on its own. Optical connectivity is being pulled further inside the package, along with silicon photonics. Packaging is up for reinvention, and testing has requirements nobody has met yet.

As semiconductor manufacturing becomes fully vertical, where things sit and how they talk to each other become open questions in every direction at once, and answering them will take decades of proposals, evaluations, successes, and failures.

In some ways, memory by design is only one major force within that. It will be shaped as much by everything happening around it as by its own roadmap. We can get tangled in the details of only the memory vertical, or just the memory-versus-logic theme, but when the whole landscape is in flux, what survives is utterly unpredictable. Every boundary that gets abolished creates a new boundary somewhere else, with a fresh set of parties on either side of it.

What makes this a long story instead of a two-year one is that the vertical direction has barely been used. The plane is exhausted, and its exhaustion took 60 years to arrive. The stack is at the beginning.

Bonding pitch, die thinning, power delivery through the structure, heat paths out of it, test after the parts are fused, and design tools capable of describing memory and logic as a single object: each of those would have been a decade of work on its own in a calmer industry, and all of them are being attempted simultaneously by companies with more capital than they have ever had and customers who will take everything they can make.

Which leaves the naming problem we started with. The flat era had a law, a scoreboard and a marketing number, and the industry knew what to call itself. What follows has none of those. The man who was lifted out of Flatland never got a vocabulary for what he saw either. He simply could not go back to describing his country the way he had.

Nilesh Jasani is the founder and CEO of GenInnov Pte Ltd Singapore. Vibhu Sharma is a veteran of the semiconductor industry and a strategic technology advisor for GenInnov. The article is republished with permission.

Sign up here to comment on Asia Times stories Or

Thank you for registering!

Share on X (Opens in new window) Share on LinkedIn (Opens in new window) LinkedI Share on Facebook (Opens in new window) Faceboo Share on WhatsApp (Opens in new window) WhatsAp Share on Reddit (Opens in new window) Reddi Email a link to a friend (Opens in new window) Emai Print (Opens in new window) Prin

MENAFN11092026000159011032ID1111651327