The Memory Wall: Past, Present, and Future of DRAM
The article highlights the challenges facing DRAM, including slowed scaling, rising AI-driven memory demand, and high costs of HBM, while emphasizing the need for innovation and new memory technologies.
Read original article
The article discusses the current state and future prospects of Dynamic Random Access Memory (DRAM), highlighting the challenges it faces in scaling and innovation. It notes that while Moore's Law has historically applied to DRAM, its scaling has significantly slowed, with density increases dropping from a doubling every 18 months to just 2x over the past decade. This stagnation is particularly problematic as the demand for memory, driven by advancements in artificial intelligence (AI), continues to grow. High Bandwidth Memory (HBM), essential for AI applications, is becoming increasingly expensive, comprising a large portion of manufacturing costs. The article explores potential solutions to the "memory wall," including extending the HBM roadmap and developing new memory technologies like Compute-in-Memory (CIM) and 3D DRAM. It emphasizes the need for innovation in DRAM to keep pace with improvements in logic chips, which are outpacing memory advancements. The article concludes by underscoring the importance of addressing these challenges to support the evolving demands of computing.
- DRAM scaling has slowed significantly, with only a 2x increase in density over the last decade.
- The rise of AI is increasing demand for memory, particularly High Bandwidth Memory (HBM), which is costly.
- Potential solutions to the memory wall include new memory technologies and extending existing roadmaps.
- Innovation in DRAM is crucial to match the advancements in logic chips.
- The DRAM industry faces economic and performance bottlenecks that need to be addressed.
Related
Extreme Measures Needed to Scale Chips
Semiconductor technology faces challenges in scaling for AI demands. Innovations like EUV lithography and chip stacking are key. Japanese researchers explore linear accelerators for EUV light. Attracting young talent is crucial.
Fourth Workshop on DRAM Security (DRAMSec)
The Fourth Workshop on DRAM Security (DRAMSec) at ISCA 2024 addresses persistent DRAM security challenges like Rowhammer attacks, side-channel vulnerabilities, and DoS threats. Keynote speakers discuss evolving threats and future solutions. Participants can attend in person or remotely.
Extreme Measures Needed to Scale Chips
The July 2024 IEEE Spectrum issue discusses scaling compute power for AI, exploring solutions like EUV lithography, linear accelerators, and chip stacking. Industry innovates to overcome challenges and inspire talent.
The Flash Memory Lifespan Question: Why QLC May Be NAND Flash's Swan Song
Evolution of NAND Flash memory technology faces challenges with QLC NAND due to density limitations. Transition from SLC to QLC impacts write cycles and speeds. Comparison with NOR Flash, scaling issues, and pSLC cache usage in QLC SSDs are discussed. Future alternatives like 3D XPoint are considered.
Magnetic tunnel junction-based computational random-access memory
A study demonstrates computational random-access memory (CRAM) using magnetic tunnel junctions, enhancing energy efficiency and performance by enabling logic operations within memory cells, crucial for machine intelligence applications.
8 commentsThese days you can do thousands of calculations waiting for a few bytes of memory. And not only is the speed difference getting worse, but memory sizes aren't keeping up.
Guess we're not far away from compressing stuff before putting it in memory is something you'd want to do most of the time. LZ4 decompression[1] is already just a factor of a few away from memcpy speed.
DRAM isn't getting (much) cheaper. And it will continue to be the case in next 10 years. Considering there is nothing on the roadmap to suggest any breakthrough.
People old enough may remember in the late 00s and even up to mid 10s, there are words like we will get 16GB computer as baseline, or I could work with 64GB soon.
Reality is that DRAM price has fluctuate lately within the same range in the past ~13 years. It was only in 2023 the price floor broke through the $2/GB barrier.
Unless we somehow found a way that could magically shrink the capacitor by a substantial amount. Or we change the way we do programming.
Given a regular ROM structure on the silicon wafer, it would be possible to take advantage of parallel beams and design an electron beam lithography machine specifically for ROM to reduce the cost of programming. The final frontier would be to build a "wafer scale engine"-esque single wafer chip and we would have essentially reached the limits of what is achievable through clever design of silicon based semiconductors.
Related
Extreme Measures Needed to Scale Chips
Semiconductor technology faces challenges in scaling for AI demands. Innovations like EUV lithography and chip stacking are key. Japanese researchers explore linear accelerators for EUV light. Attracting young talent is crucial.
Fourth Workshop on DRAM Security (DRAMSec)
The Fourth Workshop on DRAM Security (DRAMSec) at ISCA 2024 addresses persistent DRAM security challenges like Rowhammer attacks, side-channel vulnerabilities, and DoS threats. Keynote speakers discuss evolving threats and future solutions. Participants can attend in person or remotely.
Extreme Measures Needed to Scale Chips
The July 2024 IEEE Spectrum issue discusses scaling compute power for AI, exploring solutions like EUV lithography, linear accelerators, and chip stacking. Industry innovates to overcome challenges and inspire talent.
The Flash Memory Lifespan Question: Why QLC May Be NAND Flash's Swan Song
Evolution of NAND Flash memory technology faces challenges with QLC NAND due to density limitations. Transition from SLC to QLC impacts write cycles and speeds. Comparison with NOR Flash, scaling issues, and pSLC cache usage in QLC SSDs are discussed. Future alternatives like 3D XPoint are considered.
Magnetic tunnel junction-based computational random-access memory
A study demonstrates computational random-access memory (CRAM) using magnetic tunnel junctions, enhancing energy efficiency and performance by enabling logic operations within memory cells, crucial for machine intelligence applications.