Brian Kernighan on the origins of Unix

Brian Kernighan on the origins of Unix

Cgroups have two things that a virtual machine setup does not have:

1. Limits to determine which containers swap more than others. By setting memory.low to your "fair" share without overcommiting, and memory.high to your limit with overcommit, you bias the kernel's paging to page out memory hogs (both to swap and paging out their executable pages), and to slow down processes that can't reclaim memory to stay in their lane.
2. An interface for a management process to use to identify cgroups that are using more than their fair share. You can monitor memory.events for changes, and use the notifications to tell containers to reduce their memory use.

Now, if you're dealing with people who overcommit physical RAM but who've not read and understood In defence of swap, you're in trouble anyway - they're working on a faulty model of how the OS allocates memory to processes to begin with, and both models are going to break because they're expecting "magic" to save them. But with competent people, cgroups already does better than VMs; and because cgroups has more visibility into what the workload is doing, it's more likely that cgroups will improve over time.

As an example, EPOC32 had low memory signals it sent to the applications to let them know that the system was low on memory, and that it was out of memory. Applications were expected to handle these signals by discarding caches, performing an early GC etc - and could also (ab)use them for better performance (e.g. don't run GC until the system is low on memory). You can get similar signals in a cgroup-aware application by having the application monitor memory.events, and treating a breach of the "low" threshold as "memory is running low" and a breach of the "high" threshold as "out of memory". Then, your JVM could be aware of this, and whenever it breaches the "low" threshold (container using too much memory), it does a GC run, while when it breaches the "high" threshold, it also discards any cached JITted sections etc.

This interacts in a good way with the kernel's preference to swap stuff that breaches low more than it swaps stuff that breaches high - when you breach low, you try to come back under control, when you breach high, you try to avoid being killed.