An immensely powerful processor can spend its time waiting. In artificial intelligence servers, billions of transistors are only fully useful if data arrives fast enough. That is why a specialized type of memory, HBM, has become as strategically important as the accelerators it supports. And why the AI race is also playing out in assembly plants, far from chatbot demonstrations.
Looking ahead to September 2026, this dependence provides an essential framework for understanding the sector. The industrial facts presented here draw on announcements and trends established through 2024; their implications for 2026 are forward-looking analysis, not an account of subsequent events.
The real challenge: feeding the machine
Training a large model involves performing vast numbers of operations on enormous sets of numbers: parameters, activations and gradients. Running it afterward, during inference, still requires moving data. Yet accelerators’ arithmetic power has advanced faster than systems’ ability to supply them with those numbers. This is the “memory wall”: adding computing power is no longer enough.
HBM, short for High Bandwidth Memory, addresses this problem through a distinctive architecture. Multiple layers of DRAM are stacked and linked vertically by connections that pass through the silicon. These stacks sit as close as possible to the processor, within a shared package, rather than communicating from components scattered across a circuit board.
The benefit comes in particular from a very wide interface: large amounts of data travel in parallel over short distances. Imagine several hundred modest lanes rather than a handful of roads that must keep getting faster. This proximity also improves the energy efficiency of transfers. It does not, however, eliminate access times or power constraints.
Bandwidth and capacity: two different battles
Bandwidth measures the amount of data that can be transferred per second. Capacity refers to how much the memory can hold. AI needs both. A model that is too large must be spread across multiple accelerators or rely on more distant memory. In either case, the additional transfers can complicate execution and reduce performance.
Inference illustrates this tension particularly well. When a model generates a response, it rereads its parameters and manipulates a cache tied to the conversation’s context. When few requests are processed simultaneously, generation can be limited primarily by memory traffic. Batching requests improves compute utilization, but also increases capacity requirements and can complicate latency management.
More HBM therefore does not automatically make every application faster. The outcome depends on the model, numerical precision, software and workload. But insufficient memory can prevent an accelerator that looks powerful on paper from realizing its potential.
Packaging: the invisible infrastructure
Bringing memory and processor closer together requires far more than soldering. They must be connected through a structure offering an extremely high density of links, often a silicon interposer. This so-called “advanced” assembly combines distinct components in a single package. It turns packaging, long portrayed as a relatively routine final step, into a decisive technology.
TSMC’s CoWoS process has become emblematic of this shift. As early as 2023 and 2024, constraints on its capacity accompanied the surge in demand for AI accelerators. The challenge was therefore not simply to manufacture more compute chips: they also had to be packaged with their memory, tested and produced in sufficient working quantities.
Architectures announced in 2024, notably Nvidia’s Blackwell, along with AMD’s previously unveiled MI300 accelerators, illustrate this growing complexity. Performance rests on a system combining compute, memory and interconnects. Comparing only process nodes or transistor counts is like judging a railway station without looking at its tracks.
An industrial supply chain with bottlenecks
HBM production is concentrated around three major memory manufacturers: SK hynix, Samsung and Micron. But their products are not interchangeable overnight. Each generation must be qualified for an accelerator and its package, covering electrical performance, temperature, reliability and mechanical compatibility. An available supplier is not necessarily one whose products can be used immediately.
In 2024, the HBM3E production ramp was already showing that the pace depended on several areas of expertise. Manufacturing memory chips was not enough: they had to be thinned, stacked and interconnected, with yields kept under control. Increasing the number of layers boosts capacity, but also heightens thermal and mechanical challenges.
- Memory manufacturers must make trade-offs in allocating investment, equipment and production space.
- Packaging specialists must expand assembly capacity without sacrificing yields.
- Suppliers of substrates, materials and testing services must keep pace.
The risk is a missing link. Processors may be ready while qualified memory or packaging capacity is not. Conversely, additional capacity brought online too late may arrive after orders have been revised. Investment announcements therefore do not immediately translate into delivered servers.
The price of AI is also decided here
HBM is expensive to produce and requires sophisticated packaging. When a defect emerges late in the process, it can hold up or compromise an entire set of costly components. Intermediate testing and yield control become essential. These economics favor companies able to finance large volumes and secure supplies well in advance.
For a cloud provider, however, the bill does not end with the purchase. Memory capacity better matched to requirements can make it possible to serve more requests or use fewer accelerators for a given model. The relevant metric becomes the cost of delivering the service, with its latency and quality requirements, rather than the price of an individual chip.
This dependence also reveals a concentrated geography: South Korean and American memory manufacturers, advanced packaging closely tied to Taiwan, and equipment and materials sourced from several countries. Diversifying locations reduces certain risks without quickly recreating the entire ecosystem. An additional processor factory alone does not guarantee industrial self-sufficiency in AI.
Software can ease the pressure
The answer is not exclusively hardware-based. Quantization, cache compression and better organization of computations reduce the amount of data that must be stored or moved. Some accelerators also favor alternative memory hierarchies. These approaches can limit reliance on HBM for specific uses, with trade-offs in precision, flexibility or cost.
What comes next? For September 2026 and beyond, a plausible scenario is a parallel race to increase memory capacity, bandwidth and packaging yields. Its outcome will depend as much on software advances and actual demand as on new factories. To understand who can deliver AI, it will be necessary to look behind the star processor: at its memory stacks, their suppliers and the production lines capable of bringing it all together.
