Return to Answer

I've found two explanations(of the same thing) as to why kswapd0 does constant disk reading happens well before OOM-killer kills the offending process:

1. see the answer and comment of this askubuntu SE answer
2. see the answer and David Schwartz's comments of this answer on unix SE

I'll quote here the comment from 1. which really opened my eyes as to why I was getting constant disk reading while everything was frozen:

For example, consider a case where you have zero swap and system is nearly running out of RAM. The kernel will take memory from e.g. Firefox (it can do this because Firefox is running executable code that has been loaded from disk - the code can be loaded from disk again if needed). If Firefox then needs to access that RAM again N seconds later, the CPU generates "hard fault" which forces Linux to free some RAM (e.g. take some RAM from another process), load the missing data from disk and then allow Firefox to continue as usual. This is pretty similar to normal swapping and kswapd0 does it. – Mikko Rantalainen Feb 15 at 13:08

If anyone has a way as to how to disable this behavior(maybe recompile kernel with what options?), please let me know as soon as possible! Much appreciated, thanks!
EDIT: I've just found out and tested that vm.overcommit_memory=2 avoids the disk thrashing that leads to frozen OS (even though the OOM-killer doesn't get a change to trigger, which makes sense because it only triggered well after the disk-thrashing anyway, with vm.overcommit_memory=0) like: cc1plus: out of memory allocating 127440 bytes after a total of 897024 bytes
EDIT2 ok the above worked for me with the default vm.overcommit_ratio=50 but if I set it to 200 then the disk-thrashing is back!
EDIT3 ignore these 2 EDITs, they are not the way to fix this because processes that wouldn't have died before die now sooner, also vm.overcommit_ratio=0 will cause everything to die, or something. I'll try to find a better way, likely needing kernel recompile and I'm looking into the relevant GFP flags...
EDIT4: I found a way, through patching kernel, that works for me; see the patch inside this question.

I've found two explanations(of the same thing) as to why kswapd0 does constant disk reading happens well before OOM-killer kills the offending process:

1. see the answer and comment of this askubuntu SE answer
2. see the answer and David Schwartz's comments of this answer on unix SE

I'll quote here the comment from 1. which really opened my eyes as to why I was getting constant disk reading while everything was frozen:

For example, consider a case where you have zero swap and system is nearly running out of RAM. The kernel will take memory from e.g. Firefox (it can do this because Firefox is running executable code that has been loaded from disk - the code can be loaded from disk again if needed). If Firefox then needs to access that RAM again N seconds later, the CPU generates "hard fault" which forces Linux to free some RAM (e.g. take some RAM from another process), load the missing data from disk and then allow Firefox to continue as usual. This is pretty similar to normal swapping and kswapd0 does it. – Mikko Rantalainen Feb 15 at 13:08

If anyone has a way as to how to disable this behavior(maybe recompile kernel with what options?), please let me know as soon as possible! Much appreciated, thanks! 

UPDATE: The only way I've found thus far is through patching the kernel, and it works for me with swap disabled(ie. CONFIG_SWAP is not set) but doesn't work for others with swap enabled it seems; see the patch inside this question.

I've found two explanations(of the same thing) as to why kswapd0 does constant disk reading happens well before OOM-killer kills the offending process:

1. see the answer and comment of this askubuntu SE answer
2. see the answer and David Schwartz's comments of this answer on unix SE

I'll quote here the comment from 1. which really opened my eyes as to why I was getting constant disk reading while everything was frozen:

For example, consider a case where you have zero swap and system is nearly running out of RAM. The kernel will take memory from e.g. Firefox (it can do this because Firefox is running executable code that has been loaded from disk - the code can be loaded from disk again if needed). If Firefox then needs to access that RAM again N seconds later, the CPU generates "hard fault" which forces Linux to free some RAM (e.g. take some RAM from another process), load the missing data from disk and then allow Firefox to continue as usual. This is pretty similar to normal swapping and kswapd0 does it. – Mikko Rantalainen Feb 15 at 13:08

If anyone has a way as to how to disable this behavior(maybe recompile kernel with what options?), please let me know as soon as possible! Much appreciated, thanks!
EDIT: I've just found out and tested that vm.overcommit_memory=2 avoids the disk thrashing that leads to frozen OS (even though the OOM-killer doesn't get a change to trigger, which makes sense because it only triggered well after the disk-thrashing anyway, with vm.overcommit_memory=0) like: cc1plus: out of memory allocating 127440 bytes after a total of 897024 bytes
EDIT2 ok the above worked for me with the default vm.overcommit_ratio=50 but if I set it to 200 then the disk-thrashing is back!
EDIT3 ignore these 2 EDITs, they are not the way to fix this because processes that wouldn't have died before die now sooner, also vm.overcommit_ratio=0 will cause everything to die, or something. I'll try to find a better way, likely needing kernel recompile and I'm looking into the relevant GFP flags...

I've found two explanations(of the same thing) as to why kswapd0 does constant disk reading happens well before OOM-killer kills the offending process:

1. see the answer and comment of this askubuntu SE answer
2. see the answer and David Schwartz's comments of this answer on unix SE

I'll quote here the comment from 1. which really opened my eyes as to why I was getting constant disk reading while everything was frozen:

For example, consider a case where you have zero swap and system is nearly running out of RAM. The kernel will take memory from e.g. Firefox (it can do this because Firefox is running executable code that has been loaded from disk - the code can be loaded from disk again if needed). If Firefox then needs to access that RAM again N seconds later, the CPU generates "hard fault" which forces Linux to free some RAM (e.g. take some RAM from another process), load the missing data from disk and then allow Firefox to continue as usual. This is pretty similar to normal swapping and kswapd0 does it. – Mikko Rantalainen Feb 15 at 13:08

If anyone has a way as to how to disable this behavior(maybe recompile kernel with what options?), please let me know as soon as possible! Much appreciated, thanks!
EDIT: I've just found out and tested that vm.overcommit_memory=2 avoids the disk thrashing that leads to frozen OS (even though the OOM-killer doesn't get a change to trigger, which makes sense because it only triggered well after the disk-thrashing anyway, with vm.overcommit_memory=0) like: cc1plus: out of memory allocating 127440 bytes after a total of 897024 bytes
EDIT2 ok the above worked for me with the default vm.overcommit_ratio=50 but if I set it to 200 then the disk-thrashing is back!
EDIT3 ignore these 2 EDITs, they are not the way to fix this because processes that wouldn't have died before die now sooner, also vm.overcommit_ratio=0 will cause everything to die, or something. I'll try to find a better way, likely needing kernel recompile and I'm looking into the relevant GFP flags...
EDIT4: I found a way, through patching kernel, that works for me; see the patch inside this question.

I've found two explanations(of the same thing) as to why kswapd0 does constant disk reading happens well before OOM-killer kills the offending process, and thus in effect causing the disk thrashing(constant reading):

1. see the answer and comment of this askubuntu SE answer
2. see the answer and David Schwartz's comments of this answer on unix SE

I'll quote here the comment from 1. which really opened my eyes as to why I was getting constant disk reading while everything was frozen:

For example, consider a case where you have zero swap and system is nearly running out of RAM. The kernel will take memory from e.g. Firefox (it can do this because Firefox is running executable code that has been loaded from disk - the code can be loaded from disk again if needed). If Firefox then needs to access that RAM again N seconds later, the CPU generates "hard fault" which forces Linux to free some RAM (e.g. take some RAM from another process), load the missing data from disk and then allow Firefox to continue as usual. This is pretty similar to normal swapping and kswapd0 does it. – Mikko Rantalainen Feb 15 at 13:08

If anyone has a way as to how to disable this behavior(maybe recompile kernel with what options?), please let me know as soon as possible! Much appreciated, thanks!
EDIT: I've just found out and tested that vm.overcommit_memory=2 avoids the disk thrashing that leads to frozen OS (even though the OOM-killer doesn't get a change to trigger, which makes sense because it only triggered well after the disk-thrashing anyway, with vm.overcommit_memory=0) like: cc1plus: out of memory allocating 127440 bytes after a total of 897024 bytes EDIT2 ok the above worked for me with the default vm.overcommit_ratio=50 but if I set it to 200 then the disk-thrashing is back!

I've found two explanations(of the same thing) as to why kswapd0 does constant disk reading happens well before OOM-killer kills the offending process:

1. see the answer and comment of this askubuntu SE answer
2. see the answer and David Schwartz's comments of this answer on unix SE

I'll quote here the comment from 1. which really opened my eyes as to why I was getting constant disk reading while everything was frozen:

For example, consider a case where you have zero swap and system is nearly running out of RAM. The kernel will take memory from e.g. Firefox (it can do this because Firefox is running executable code that has been loaded from disk - the code can be loaded from disk again if needed). If Firefox then needs to access that RAM again N seconds later, the CPU generates "hard fault" which forces Linux to free some RAM (e.g. take some RAM from another process), load the missing data from disk and then allow Firefox to continue as usual. This is pretty similar to normal swapping and kswapd0 does it. – Mikko Rantalainen Feb 15 at 13:08

If anyone has a way as to how to disable this behavior(maybe recompile kernel with what options?), please let me know as soon as possible! Much appreciated, thanks!
EDIT: I've just found out and tested that vm.overcommit_memory=2 avoids the disk thrashing that leads to frozen OS (even though the OOM-killer doesn't get a change to trigger, which makes sense because it only triggered well after the disk-thrashing anyway, with vm.overcommit_memory=0) like: cc1plus: out of memory allocating 127440 bytes after a total of 897024 bytes
EDIT2 ok the above worked for me with the default vm.overcommit_ratio=50 but if I set it to 200 then the disk-thrashing is back!
EDIT3 ignore these 2 EDITs, they are not the way to fix this because processes that wouldn't have died before die now sooner, also vm.overcommit_ratio=0 will cause everything to die, or something. I'll try to find a better way, likely needing kernel recompile and I'm looking into the relevant GFP flags...