Why the current DRAM architecture synchronizes its timing in multiple banks?

Question

While I'm studying the DRAM architecture for my processing-in-memory research, the current architecture groups multiple banks into a DRAM controller, and synchronizes the timing between these banks.

For example, HBM groups its DRAM banks vertically into a vault, and the DRAM controller manages the timing into a vault manner.

Is making DRAM banks with independent memory timing not feasible due to the overhead?

Or are there any complex reasons such as electronic interference, cost, thermal problem, etc.?

Answer 1

That's the difference between bank vs. channel.

As an example, HBM2 has 8 channels and each channel has 16 banks. As you know, different channels can be run independently (there's no timing restriction among different channels). But, banks are dependent each other, so there are many timing restrictions when you access different banks in a same channel.

You're asking why HBM2 can't be designed as it has 128 channels where each channel has 1 bank. That's because there's area penalty in the latter.

Those banks in a same bank are sharing some resources of the channel. Two major shared resources are data bus and command decoder.

Datapath: You can't read from 2 or more banks in a same channel at the same time because those banks share the same highway between themselves and the I/O pad (or TSV, or DQ pin, or microbump).

Command decoder: You can't issue commands to 2 or more banks on the same cycle, as all those banks are controlled by the same command decoder that can receive and handle only one command that goes to one back at a time.

To be able to run banks in parallel, DRAM has to have separate datapath and separate command decoder, which will multiply the datapath area and command decoder area (and other things as well, like TSVs, etc.) by factor of say 16. This will blow up the silicon area.

You can imagine this way:

You own a hotel where each hotel has 16 towers (like Hilton). (This is like 1 channels X 16 banks/channel.) However, 16 towers share the same lobby, front desk, front door, valet and parking entrance.

Now you say "I want guests can concurrently check in, park and go in and out without any congestion as if those towers are like separate hotels".

To meet that requirement, your architect comes back with a new design where it's a giant hotel that has 16 separate lobbies, front desks, front doors, valets and parking entrances. And the size of the new hotel would be about 16 times of size of a hotel having one elevator. The total land and cost to build the giant hotel will be almost equal to the land and cost to build the 16 hotels with 1 elevator. (This is like 16 channels X 1 bank/channel).

Now you can see what the area difference is between those two cases.