5

I have a custom program that performs analysis on some data. This is a very memory-hungry operation that can potentially use thousands of gigabytes of RAM. My computer only has 16gb of RAM, so my only solution is to use swap memory on large hard drives.

I managed to do this already and it worked well, however the process eats up all of my real RAM first before starting to use the swap space. This makes my computer completely unusable during the very long time it takes to execute the operation. It also makes it difficult to monitor the operation because everything becomes extremely slow and unresponsive.

So is there a way in Linux to force a single process to only use memory in swap space? (without having to implement a custom allocator)

Ideally, I could leave the program running in the background (potentially for days or weeks) while still having access to the computer. The benefit of an extra ~16gb of RAM to the program seems hardly worth the loss of a computer for multiple days.

Improve this question
5
  • 2
    The kernel handles this, you can't redirect certain applications to swap and others not to use swap, the kernel decides that based on it's activity. When an application is in "swap" it is not doing anything, it is waiting for real RAM so it can do something (it is "swapped" out of RAM and put in a holding area, then swapped back when the kernel decides its time for it to run). The real problem here is your "custom program" has no limits on what it can do, whoever programmed it should set it up to have limits.
    – acejavelin
    Commented Oct 23, 2020 at 20:14
  • 3
    serverfault.com/a/344187
    – Kamil Maciorowski
    Commented Oct 23, 2020 at 20:14
  • lol... I stand corrected... Somewhat. ;)
    – acejavelin
    Commented Oct 23, 2020 at 20:15
  • 1
    You're asking a XY question, as well as misunderstand how swap and virtual memory works. The solution to your actual problem probably lies in adjusting the (execution) priority of your process (aka 'nice' command).
    – sawdust
    Commented Oct 23, 2020 at 20:27
  • 1
    don't use swap. Use zram instead and you'll soon see the difference in such memory-hungry situations
    – phuclv
    Commented Oct 23, 2020 at 23:33

1 Answer 1

Reset to default
6

So is there a way in Linux to force a single process to only use memory in swap space? (without having to implement a custom allocator)

No, this is not possible. The process cannot run on the swap; it must always first load the memory pages from the swap to the physical memory and then the process can continue running.

So what you need is to tell the system to limit the resources of the process, e.g. via cgroups, as suggested by Kamil's comment, or I would try prlimit (1), to set the first one or both of these two limits:

  • Maximum Resident Set Size (RSS) - how much memory in total process has in the physical RAM
  • Maximum locked-in-memory address space - how much memory can be fixed in RAM to not be swapped out.
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .