[linux-mm-cc] swap-device write failure under low memory

[Uma shankar]

Hello nitin,

Occasionally, xvMalloc will try to grow the pool.
This memory allocation can fail under low memory.

This will be informed to the kernel as a "device write" failure.
The page which was being written will not be reclaimed, but
the kernel will continue to try swap out of other pages ( as kernel
thinks that swap has free space. )

Wont this lead to the reclaim code ( kswapd or the direct reclaim
path ) hogging the processor for some time before OOM is
finally announced ?

And what if a NOFAIL allocation attempt results in this ?

Have you analysed this scenario ?

rgds,
shankar

[Nitin Gupta]

Hi,

Post by Uma shankar
Occasionally, xvMalloc will try to grow the pool.
This memory allocation can fail under low memory.
This will be informed to the kernel as a "device write" failure.
The page which was being written will not be reclaimed, but
the kernel will continue to try swap out of other pages ( as kernel
thinks that swap has free space. )
Wont this lead to the reclaim code ( kswapd or the direct reclaim
path ) hogging the processor for some time before OOM is
finally announced ?
And what if a NOFAIL allocation attempt results in this ?
Have you analysed this scenario ?

If you ever get too many write failures, its an indication that you
have oversized ramzswap device. From what I have observed, swap write
failure quickly lead to system hang. In ideal case, ramzswap should
be able to dynamically resize based on cache hit-rate, system memory
pressure etc., but all this is not yet done. So, for now, best would
be to experiment a bit and get some idea of ramzswap disksize that helps
your workload while still not getting into messy conditions like OOM,
swap write failures.

Thanks,
Nitin

[Uma shankar]

Post by Nitin Gupta
If you ever get too many write failures, its an indication that you
have oversized ramzswap device. From what I have observed, swap write
failure quickly lead to system hang. In ideal case, ramzswap should
be able to dynamically resize based on cache hit-rate, system memory
pressure etc., but all this is not yet done. So, for now, best would
be to experiment a bit and get some idea of ramzswap disksize that helps
your workload while still not getting into messy conditions like OOM,
swap write failures.
Thanks,
Nitin

And also there is a small uncertainty of how many pages will be compressable.
A large Tmpfs file may get overcommitted at the time of initial
creation ( because most pages
are compressable at that time) , but may later induce OOM after a
later file-data-update
if many pages become uncompressable after the update. ( As of now,
75% of page-size
is the threshold in ramzswap. ).

I guess the problems with a large disksize value may be told to the
users in the
Readme file.

[Uma shankar]

sorry for the bad formatting......

The ng server has reformatted my post......