Compressed swap [LWN.net]

Compressed swap

Posted Mar 28, 2014 22:31 UTC (Fri) by nybble41 (subscriber, #55106)
In reply to: Compressed swap by gfa
Parent article: Compressed swap

> it would need changes on the code which retrieves the pages from swap...
> you wouldn't be able to mix zswap and regular swap on the same system

Why is that? What I had in mind seemed fairly simple to integrate, though I'm not a Linux kernel developer and may be missing something. The idea was that when frontswap hands a page to zswap and there isn't enough room for it in zswap's RAM, zswap picks one of its less-recently-used compressed pages and hands it off to the normal swap code, storing whatever token it gets back to identify the page for later retrieval. That frees up the page to store the new compressed data. Later, if and when frontswap asks for one of the old pages' data back, zswap can request the compressed page back from the normal swap subsystem and handle the request as it normally would.

to post comments

Compressed swap

Posted Apr 14, 2014 2:30 UTC (Mon) by kmeyer (subscriber, #50720) [Link] (1 responses)

> The idea was that when frontswap hands a page to zswap and there isn't enough room for it in zswap's RAM, zswap picks one of its less-recently-used compressed pages and hands it off to the normal swap code, storing whatever token it gets back to identify the page for later retrieval.

Usually (I think) the page (a pointer in virtual memory) is unmapped from the page table and swapped out. Then when that pointer is accessed, a hardware fault occurs because the VMA is unmapped, and the kernel fault handler then brings back the page from swap.

> That frees up the page to store the new compressed data. Later, if and when frontswap asks for one of the old pages' data back, zswap can request the compressed page back from the normal swap subsystem and handle the request as it normally would.

So, the way this would actually be implemented (I think) is allowing pages in zswap's backing store to be swapped, but only when requested by zswap's front-end.

Because you don't want to accidentally access those backing pages and bring them back in and trigger more swapping elsewhere, this would require some additional complexity in zswap to track which backing pages are actually available and which need to be paged in.

You might also have some terrible / recursive locking and code flow going on between the memory request -> no clean pages -> need to swap a page -> pager <-> zswap <-> pager again(!); I'm unfamiliar with the area but it sounds like a nightmare (that zswap likely tries very hard to avoid right now with zbud).

It may be possible, but it would probably be a pain in the ass to implement and would have room for latent bugs.

Compressed swap

Posted Apr 14, 2014 22:36 UTC (Mon) by nybble41 (subscriber, #55106) [Link]

> ... this would require some additional complexity in zswap to track which backing pages are actually available and which need to be paged in.

I agree that zswap would need to keep track of its pages. However, there shouldn't be any risk of accidentally triggering swapping. The compressed pages would only be read from swap at zswap's request.

> You might also have some terrible / recursive locking and code flow going on ...

I don't think so. The requests should be unidirectional, not recursive. The zswap RAM would be reserved and managed by zswap, not the pager. This assumes that zswap can access the swap interface directly the same way the pager does; if the pager and swap are tightly integrated then some refactoring would be in order.

Memory request -> no clean pages -> pager -> zswap -> write existing page directly to swap -> compress new page -> free source page -> resume thread.

Page fault -> pager -> zswap -> read compressed page back from swap into preallocated buffer -> decompress into target RAM -> resume thread.