An ultra-powerful processor waiting for its data is like a restaurant kitchen cut off from deliveries: the equipment is there, but nothing comes out. Behind the race for artificial intelligence chips, a less visible battle is unfolding over memory and assembly. It helps explain why producing more processors is not enough to make computing abundant and affordable. Looking ahead to September 2026, the truly strategic product is a complete system, not just a chip.
This analysis draws on established technologies and public announcements, particularly those from 2024. The outlook for September 2026 is presented as projections, not as an account of events verified as of that date.
The processor computes, the memory feeds it
To train a model or have it generate a response, an accelerator must handle its parameters and intermediate data. Some of this information resides in memory outside the processor, but positioned very close to it. Every transfer takes time and consumes energy. If the compute units request data faster than the memory can supply it, they wait.
This is the “memory wall” problem. Adding compute units therefore does not guarantee a proportional speedup. In some tasks, performance depends more on the available data throughput than on the number of operations theoretically possible. The spectacular figures advertised for processors thus tell only part of the story.
Two characteristics must be distinguished. Capacity, expressed in gigabytes, determines what can be kept close to the computation. Bandwidth measures the amount of data that can be transferred per second. A large reservoir is not necessarily fast; a very fast connection does not compensate for memory that is too small.
HBM: layers of memory to ease bottlenecks
High-bandwidth memory, or HBM, addresses this challenge by stacking multiple layers of DRAM. Vertical connections run through the layers, while a very wide interface connects these stacks to the processor. The aim is to move large amounts of data simultaneously over short distances, rather than drive a limited number of connections ever faster.
HBM did not originate with generative AI. But generative AI has made it a crucial component of high-end accelerators. The HBM3 and then HBM3E generations have supported this growth. In 2024, announcements around Nvidia Blackwell and AMD Instinct MI325X already illustrated the importance placed on memory capacity and bandwidth.
The trade-offs remain demanding. Stacking more layers complicates manufacturing, connections and thermal management. A faulty stack does not become usable simply because the neighboring processor works perfectly. Component qualification, manufacturing yields and integration matter as much as performance on a specification sheet.
Packaging is no longer just a wrapper
In electronics, packaging refers to integrating and connecting chips within a usable package. For AI accelerators, this stage has become a leading technology in its own right. The processor and several HBM stacks must be placed side by side, with thousands of short connections, while ensuring adequate power delivery and cooling.
Processes such as TSMC’s CoWoS make use of an interposer, an intermediate structure that enables very dense connections. This integration takes different forms depending on the architecture. The principle remains the same: bring components closer together to create a system that performs better than chips scattered across a board.
In 2023 and 2024, pressure on advanced packaging capacity showed that the bottleneck could lie downstream of processor manufacturing. Having more fabricated silicon does not solve a shortage of assembly capacity. And a new line does not become productive instantly: it requires equipment, expertise and stable yields.
An industrial supply chain with multiple constraints
The DRAM market is dominated by Samsung, SK hynix and Micron. In HBM, however, the challenge goes beyond volume: manufacturers must successfully develop a given generation, deliver components that meet customer requirements and secure their qualification for a specific accelerator. Replacing a supplier is therefore not like swapping a memory module in a computer.
Producing an accelerator depends on several timelines:
- Logic: manufacturing the processor using the chosen process.
- Memory: producing and testing suitable HBM stacks.
- Integration: securing interposers, substrates and assembly capacity.
- The system: delivering compatible boards, servers, networking and cooling.
This interdependence gives purchase commitments and capacity reservations strategic value. It also explains why an investment announcement does not immediately translate into deliveries. Looking ahead to 2026, the plausible scenario is one of varying pressures across generations and manufacturing stages, rather than a uniform shortage of all AI chips.
Why the bill is not determined by computing power alone
HBM and its integration contribute significantly to accelerator costs. They involve complex processes, testing and high-value-added assembly. When several expensive components are brought together, avoiding the loss of the entire package because of a defect becomes paramount. Without a detailed public bill of materials, assigning a universal percentage of the price to memory would be misleading.
For users, however, the right metric is not simply the purchase price: it is the cost of the work performed. A more expensive accelerator can be cost-effective if it handles more requests or allows a model to be kept on fewer machines. Conversely, impressive computing capacity that is poorly supplied with data amounts to paying for underused resources.
Inference makes this issue particularly tangible. During generation, some workloads are heavily constrained by data movement. Long conversations also require retaining a cache of information, often called a KV cache, whose size grows with the context and the number of simultaneous requests. Memory then affects both service throughput and latency.
Software can ease the squeeze
Not everything can be solved by adding HBM. Quantization reduces numerical precision to decrease memory usage and data transfers, provided quality is preserved. More efficient attention methods, better cache management and request batching can also improve hardware utilization. These gains, however, depend on the model and the service’s constraints.
What comes next? For September 2026 and beyond, new memory qualifications and integration capacity will need to be watched as closely as processor announcements. More abundant supply could ease some costs; more ambitious models could absorb those gains. The decisive question will therefore remain a practical one: how much data can a system keep close to the computation, move efficiently and process for every euro spent?
