4

I have 24GB of available RAM in my machine, but I usually use far less than 8GB of it. However, sometimes I'm involved with large data files and I suddenly need 40GB, and on infrequent occasions, I'd like to be able to access even more (up to 60GB). This is mainly using numpy and Apache Arrow.

Right now I have a 20GB ssd swap file, and that works fine, but is obviously slow. I'd like to replace it with zRAM, and assuming say I get double compression on 16GB, that's 32GB, plus the leftover 8GB = 40GB total in "ram". Ideally though, I'd like to go even bigger than 40GB on occasion, and wonder if it's possible for the stuff that doesn't fit into zRAM to spillover further into a "traditional" SSD swap file or partition.

Is this doable? Obviously I'd want the zRAM to be prioritised over the disk swapfile.

Thomas Browne
  • 376
  • 3
  • 8
  • 26

2 Answers2

8

zram

zram essentially is storage of compressed swap data in memory rather than on a disk. It acts as a regular and separate swap device. It thus can be used as additional swap space, besides swap partitions or swap files, and be given a certain priority.

Priorities for swap can be set with the swapon tool, and can be defined in /etc/fstab with the pri=value option. See man swapon.

Consider also zswap

Instead of using zram, you may consider another technology, zswap. Here, also memory is used to store swapped data. However, with this mechanism, your memory acts as a compressed memory cache for swap. Data that needs to be cached is first stored in zswap, and only committed to disk if needed, speeding up swapping and reducing disk I/O.

vanadium
  • 97,564
2

Or not using swap on disk any more: https://docs.kernel.org/admin-guide/blockdev/zram.html#writeback

With CONFIG_ZRAM_WRITEBACK, zram can write idle/incompressible page to backing storage rather than keeping it in memory. To use the feature, admin should set up backing device via:

echo /dev/sda5 > /sys/block/zramX/backing_dev

before disksize setting. It supports only partitions at this moment.

It will take whole partition as a fallback storage of zram like swap partition.

Or even on file: https://unix.stackexchange.com/questions/581243/using-loopdev-for-zram-writeback-feature

n0099
  • 221