Linux Memory Overcommit (2007)

If you want to optimize for throughout one option is to overcommit resources (CPU/MEM/IO/NET) but utilize backpressure mechanisms to reduce load during times of saturation.

Kubernetes does this through node pressure eviction but it is pretty easy to hook into the pressure stall information and have the application handle this as well (for example, start returning 429 HTTP responses when PSI goes over a certain level).

At the end of the day the optimal overcommit is workload dependent — good metrics and an iterative approach is needed.

As then as now, you can manage it.

vm.overcommit_memory == 0 heuristic overcommit

vm.overcommit_memory == 1 (full overcommit) allows allocating more memory than there is ram + swap.

vm.overcommit_memory == 2 never overcommit. There must be enough physical ram or virtual swap to allocate the memory.

Reminder: if overcommit as a concept seems distasteful, the real ire should be directed at the Unix fork syscall, an API that will always fail on a process using over 50% of the available memory without overcommit. Ideally apps would use vfork or posix_spawn instead, but that isn't the world we live in. Overcommit is a sensible way to help apps that use a lot of memory "just work". You can always turn it off if you don't need apps using a lot of memory to be able to fork.

The truth is that if you're having issues with default overcommit configuration (namely overcommit_memory == 0 and overcommit_ratio == 50) your application probably sucks and you have to actually diagnose it and fix it.

"We run a pretty java-heavy environment, with multiple large JVMs configured per host. The problem is that the heap sizes have been getting larger, and we were running in an overcommitted situation and did not realize it. The JVMs would all start up and malloc() their large heaps, and then at some later time once enough of the heaps were actually used, the OOM killer would kick in and more or less randomly off one of our JVMs."

I know that 2007 may have been different times, but I'd argue that max heaps for all your JVMs running on a system probably shouldn't exceed around 88% of the total system memory. (percentage goes up as the total system memory goes up from 128GB -> 256GB -> 512GB)

I have been a linux user/dev for a while now, and it seems I have the wrong idea of what is memory overcommit:

For me, for a modern large CPU implementation, it simplifies a lot the userland software stack to be able to book huge virtual address spaces in userland, which are populated on page faults. If really needed, I can "release" some memory ranges of this memory virtual address space (linux munmap has some options related to just that).

I do realize I am working more and more in a 'infinite memory model' (modulo some "released" ranges from time to time).

For instance, I would be more that happy to book a few 64GiB virtual address ranges on my small 8GiB workstation.

I am talking about anonymous memory, not file backed mmaping which is another story.

The problem with overcommitting beyond the sum of RAM and swap is that userspace processes can't protect themselves against the kernel's lies. You allocate memory, handle the result, access the address space and someone dies. The Linux default behaviour is an insane over-optimisation, but without it forking large processes becomes painfully expensive. The proper solution is to spawn directly through suitable system calls instead of the traditional fork() + execve() way.

Imo a good compromise to this day is to overcommit RAM with swap as fallback for correctness even if performance falls of a cliff if you use it. All of that is on top of resource limits and usage monitoring.

Is any of this still applicable in 2024?

One has to doubt...