musa sadık ünal

Memory Tiering

2024-10-20 |

This is a small write-up on memory tiering, and why we need memory tiering at all.

In today’s data centers applications and workloads are getting much larger. And in-memory applications are becoming much more common. So there is a huge demand for memory. To meet these demands datacenters are trying to expand their memory.

Let’s give a close look to hardware. An obvious solution would be we can put as much as DRAM into motherboard. Then we can use large block devices like SSD as a swap device. However, to access a page in a swap area, you need to take a page-fault (expensive) and this page will be fetched from SSD to dram.

Maybe we can put more DIMM slots into the motherboard while manufacturing? However this is not quite possible due to hardware limitations. Currently DDR4 and DDR5 requires 288 pins. And increasing the pin count on CPU is super expensive. So this is not a good solution.

Maybe we can increase the capacity of devices ? So instead using DRAM maybe we can replace with some DCPMMs. They have a higher capacity. They can be accessed through load/store instructions and they’re byte addressable. However one thing is DCPMM slower than DRAMs.

Can we do more? Also now thanks to new CXL protocol, now it’s possible to expand main memory through PCIe bus. We need to plug a CXL expander and we can use different type of devices through CXL. Again we will able to use load/store semantics.

This enables us to create tiers between devices. We able to make trade-off between different parameters:

  • Latency
  • Price
  • Bandwidth
  • Energy
  • Capacity

How to find hot pages

We would like to collect information about hot pages dynamically. Because applications have different phases, and their working set is tend to be different. There are several approaches for that:

  • Page unmapping : Intentionally unmap some pages, and accessing these pages will going to create a minor-page faults and we can infer which pages are being accessed.
  • Page table scanning : Scan processes' page table to check accessed bit.
  • Hardware sampling : Use PEBS to collect information about which pages are being accessed (Trace LLC misses).
This was the end of the blog post. You can reach me via email umusasadik at gmail com