Memory and HBM

An AI accelerator is a machine for moving numbers past arithmetic units, and arithmetic got cheap faster than the moving did. The US export control office notes that a processor cannot reach its full capabilities unless its memory capacity and bandwidth rise with its speed^[1].

How it works

Each layer in the cube is a DRAM chip, ground thin and bonded to the one below through microbumps, tiny metal bumps, and the whole stack sits on a base die that manages it. The copper-filled holes are TSVs, through-silicon vias, and they let a signal reach any layer without a wire around the outside.

An HBM3E cube talks to the processor over 1,024 wires at once, a 1,024-bit interface, sixteen times wider than a standard DDR5 memory module, the memory stick in a PC^[2]. HBM4 doubles that to 2,048 wires^[3]. Ordinary memory cannot be made that wide, because the pins would not fit on a circuit board. The stack can, since its wires run down through the TSVs and across a silicon interposer to the processor beside it, with no board in the way^[2].

Inside a memory stack

1. Base chip. A memory stack sits beside the GPU on a shared silicon base. It starts with a base chip, which controls the memory chips above it.
2. Thin. Each memory chip is ground to about 30 micrometers, thinner than a human hair.
3. Stack. Twelve chips are joined by dots of solder, and copper-filled holes run straight down through all of them.
4. Height. The stack has to fit under the cooling plate. Twelve chips reach 0.72 mm, the old limit. HBM4 raised the limit to 775 micrometers and fits up to sixteen chips.
5. Width. An HBM3E stack talks to the processor over 1,024 wires at once, sixteen times as many as a PC memory module. HBM4 doubles that to 2,048.

Simplified cross-section. Heights follow the published limits and are drawn to one vertical scale; widths are not to scale.

Answering a prompt is what the width is for. Training reuses each stored number across a huge multiplication, so a chip can fetch little and compute a lot. To generate one word of an answer, the chip reads the whole model from memory, and the whole record of the conversation so far, does a little arithmetic, and waits for the next read. That record grows with the length of the conversation and the number of users served at once. So memory size and speed usually set how many people one chip can serve.

Variants and trade-offs

HBM3E

Micron ships the high-volume parts, 24 GB with eight chips stacked and 36 GB with twelve^[4]. SK hynix's January 2026 market outlook says major research and brokerage analysts expect HBM3E to be about two thirds of all HBM shipments in 2026^[5].

HBM4

The standards body JEDEC published JESD270-4 in April 2025, setting the 2,048-bit interface and 2 TB/s per stack at 8 Gb/s per pin^[3]. SK hynix says it was first to complete HBM4 development, in September 2025, at over 10 Gb/s per pin^[6]. Samsung says it began mass production and shipped the first commercial HBM4 in February 2026, at 11.7 Gb/s per pin, with a 4 nm logic base die^[7]. Micron's parts run above 11 Gb/s for more than 2.8 TB/s per stack^[8].

The chip at the bottom of the stack is a base die, and SK hynix, which made its own base dies up to HBM3E, chose TSMC's advanced logic process for the HBM4 base die so it can hold more functions^[9]. That gives TSMC a role inside a product it does not make.

Peak bandwidth per HBM stackGB/s

Peak bandwidth per HBM stack

HBM3E, 1,024-bit	1,200 GB/s
HBM4, 2,048-bit	2,000 GB/s
HBM4, advanced configurations	3,300 GB/s

Source: Siemens EDA, HBM3E and HBM4 IC design guide, April 2026

How a stack is built

• Drill and thin. Etch vertical holes, the through-silicon vias, into a memory wafer, fill them with copper, then grind the wafer from behind until the copper comes through.
• Stack and join. The chips are joined by thermocompression bonding, which connects them through metal bumps under heat and pressure and comes in several forms^[11]. SK hynix uses Advanced MR-MUF, which stacks the chips, injects a liquid protective material between them and hardens it, and the company says the process controls warping and dissipates heat well^[6]. Samsung said in February 2024 that its twelve-high HBM3E uses TC-NCF, thermocompression with a thin non-conductive film between the chips, which it expected to help limit warping as chips get thinner^[12].
• Stay under the height limit. HBM4 raised the package specification height of a finished cube from 720 micrometers to 775, still under a millimeter^[11].

Hybrid bonding is the next step and it is not ready. Pressing copper pad straight onto copper pad, with no solder and no filler, would cut the spacing between connections from about 20 micrometers to under 1, but a tech note on SK hynix's newsroom says full-scale adoption in HBM is most likely at HBM4E or HBM5, where stacks are projected to pass twenty layers^[11].

Who makes it

Ordinary computer memory, DRAM, is a three-firm oligopoly, and HBM is the same three firms reordered: SK hynix, Samsung and Micron^[13]. Nobody else has both leading-edge DRAM and a stacking line inside one company.

SK hynix was the first company to mass-produce several generations of HBM, including HBM3 and HBM3E^[13]. UBS, as quoted in SK hynix's January 2026 market outlook, expected SK hynix to take about 70 percent of the HBM4 market for Nvidia's Rubin platform in 2026^[5]. Samsung says it shipped the first commercial HBM4 in February 2026^[7]. SK hynix broke ground in August 2026 on an advanced packaging plant in West Lafayette, Indiana, an investment of over $4 billion^[14]. Micron is modernizing its DRAM and HBM production capacity in Taiwan and has broken ground on an HBM advanced packaging plant in Singapore to expand its packaging capacity from 2027^[15].

The split is on the record because SK hynix registered shares in the United States: of 2025 HBM revenue, SK hynix took 63.2 percent, Samsung 19.3 and Micron 17.4^[13]. The filing credits the research firm IDC. These shares move fast, and in 2024 the same three stood at 56.4, 37.8 and 5.8 percent^[13]. In the first quarter of 2026 they stood at 56.4, 20.5 and 23.1 percent^[13].

Global HBM revenue by supplier, 2025%

SK hynix (KR)63.2%Samsung (KR)19.3%Micron (US)17.4%

Source: SK hynix Form 424B4 prospectus, 10 July 2026, on IDC data

HBM market forecast for 2026, Bank of America$B

HBM market forecast for 2026, Bank of America

2026	54.6 $B

Source: Bank of America estimate cited in SK hynix's 2026 market outlook, January 2026

Bank of America, in an estimate SK hynix quoted in January 2026, put the 2026 HBM market at $54.6 billion, up 58 percent on 2025^[5]. SK hynix says HBM requires greater wafer input than traditional DRAM^[13]. It also says the industry-wide reallocation of production capacity to HBM has left traditional DRAM, the kind used in PCs and phones, undersupplied, which helped drive a significant price recovery in that market from the third quarter of 2025^[13].

The chokepoint

Every leading accelerator made outside China depends on Korean or American HBM. Nvidia's GB300 NVL72 carries 20 TB of HBM3E at up to 576 TB/s^[16], Rubin up to 288 GB of HBM4 per GPU at up to 22 TB/s^[17], and AMD's MI355X 288 GB of HBM3E at 8 TB/s^[18].

The Bureau of Industry and Security rule of 5 December 2024, at 89 FR 96790, added an export classification, ECCN 3A090.c, for "High bandwidth memory (HBM) having a 'memory bandwidth density' greater than 2 gigabytes per second per square millimeter". Its technical note defines that density as the memory bandwidth in gigabytes per second divided by the area of the package or stack in square millimeters. The rule also reaches stacks made outside the United States, through a foreign direct product rule that catches goods made abroad with American technology^[1]. BIS stated in the rule that all HBM stacks then in production exceeded that threshold^[1]. Under 15 CFR 742.6(b)(10)(ii) BIS reviews applications for 3A090.c to Macau or a Country Group D:5 destination such as China with a presumption of approval when neither the buyer nor its ultimate parent is headquartered in one of those places, and with a presumption of denial otherwise^[19].

Two provisions narrow the control. License Exception HBM, at 15 CFR 740.25, lets an exporter headquartered in the United States or a Country Group A:5 ally ship HBM below 3.3 GB per second per square millimeter without a license, if the designer of the co-packaged chip buys it directly and it goes straight to the packaging site^[20]. The rule also excludes co-packaged chips that combine HBM and logic and whose main function is processing, which other chip controls may cover instead^[1]. CSIS said in December 2024 that the rules still allowed HBM2 to go to China with end-use and end-user restrictions, and that Nvidia's H20, whose HBM3 is packaged with the processor, was then still exportable under the accelerator thresholds^[21].

In December 2024 CSIS called China's strongest DRAM maker, CXMT, one of only two Chinese firms with a credible path to producing advanced HBM^[21]. The Institute for Progress reported in October 2025 that CXMT was planning to produce HBM3 in 2026^[22]. Huawei said in September 2025 that its Ascend 950 chips would use two HBM designs of its own, HiBL 1.0 and HiZQ 2.0, and that the 950DT, due in the fourth quarter of 2026, would carry 144 GB of HiZQ 2.0 at 4 TB/s^[23]. The announcement did not say who would manufacture the memory^[23]. SK hynix says HBM requires advanced production processes such as TSV packaging and compliance with the rigorous testing and approval processes its customers set^[13].

Key evaluation criteria

• Bandwidth per stack sets how fast a chip can serve answers: over 1.2 TB/s on HBM3E, over 2 TB/s on HBM4^[10].
• Capacity per stack decides how large a model fits on one accelerator, up to 64 GB with sixteen chips stacked in the JEDEC HBM4 standard^[3]. Samsung said in February 2026 that its twelve-high HBM4 comes in 24 to 36 GB and that sixteen-high parts would take it to 48 GB^[7].
• Stack height, 775 micrometers in the HBM4 package specification, caps how many layers fit in a stack^[11].
• Stack yield matters because a chip that fails after stacking can scrap the whole cube. Micron sends only dies that pass testing on to assembly and tests the finished cube again^[8].
• Thermal path gets harder with every layer, and heat dissipation becomes a major challenge at 16 to 20 layers as heat builds up in the stack^[11].
• Qualification means passing each customer's tests, and SK hynix credits its experience with complex qualification processes for its standing as a trusted HBM supplier^[13].