Patrick F. Stolt

23.9 An 8-channel 4.5Gb 180GB/s 18ns-row-latency RAM for the last level cache

In recent years, the demand for memory performance has grown rapidly due to the increasing number of cores on a single CPU, along with the integration of graphics processing units and other accelerators. Caching has been a very effective way to relieve bandwidth demand and to reduce average memory latency. As shown by the cache feature table in Fig. 23.9.1, there is a big latency gap between SRAM caches in the CPU and the external DRAM main memory. As a key element for future computing systems, the last level cache (LLC) should have a high random access bandwidth, a low random access latency, a density of 1 to 8Gb, and all signal pads located on one side of the chip [1]. A logic-process-based solution was proposed [2], but it is not scalable, and has a high standby current due to its need for frequent refresh. HBM2 was also proposed [3], but its row latency is not better than conventional DRAM, and its random-access bandwidth is still limited by tFAW, as shown in Fig. 23.9.1. This paper describes the high-bandwidth low-latency (HBLL) RAM design: how it overcomes these challenges and meets requirements in a cost-effective way.

A computer designed half Gb 16-channel 819Gb/s high-bandwidth and 10ns low-latency DRAM for 3D stacked memory devices using TSVs

Presented is a novel half Gb DRAM device for 3D stacked systems utilizing TSV. It is designed through the use of a new computer-aided design methodology and which realizes 819 Gb/s bandwidth across 16 channels and <;10ns read latency on a 45nm DRAM process. The architecture is based on small subarrays with short WL and BL to realize the low latency and energy efficiency. We also integrated several circuit techniques, including adaptive power to speed-up access time and banks rotation to reduce thermal issues. The proposed device is also estimated in a system simulation that shows that the power efficiency is higher than comparable systems.

Probabilistic replacement strategies for improving the lifetimes of NVM-based caches

Non-volatile memory (NVM) technologies present an opportunity to improve area efficiency and reduce energy consumption throughout the memory hierarchy. However, write endurance can hinder the adoption of NVM in lower-level caches. With an estimated write endurance of one trillion write cycles, Spin-Torque Transfer RAM (STT-RAM) is a more likely candidate for application as an L2 cache than Resistive RAM (ReRAM) or Phase-Change Memory (PCM). In resource-constrained systems where aggressive wear-leveling techniques cannot be applied, light-weight alternatives may be necessary to extend the lifetime of the cache. In this paper, we propose and evaluate a hybrid-random replacement policy as a low-overhead approach to wear-leveling to improve the lifetime of a large non-volatile memory L2 cache. We investigate another probabilistic mechanism that utilizes approximate counters as an alternative method of injecting random events in the eviction stream. We show that our hybrid-random policy extends the lifetime of an NVM L2 cache by 0.5 to 16 years across many benchmarks over an LRU-replacement baseline. Our approximate counter approach further extends the lifetime by 1.7 to 19 years over the baseline but incurs a higher overhead.