Async hazard: MMAP is blocking IO

I like this point - it's no secret that mmap can make memory access cost the same as an IO (swap can too) - but the interaction with async schedulers isn't immediately obvious. The cost can, sometimes, be even higher than this post says, because of write back behavior in Linux.

Mmap is an interesting tool for system builders. It's super powerful, and super useful. But it's also kind of dangerous because the gap between happy case and worst case performance is so large. That makes benchmarking hard, adds to the risk of stability bugs, and complicates taming tail latency. It's behavior also varies a lot between OSs.

It's also nice to see all the data in this post. Too many systems design conversations are just dueling assertions.

IMO this is a strong argument for proper threads over async: you can try and guess what will and won't block as an async framework dev, but you'll never fully match reality and you end up wasting resources when an executor blocks when you weren't expecting.

> How do other mmap/madvise options influence this (for instance, MADV_SEQUENTIAL, MADV_WILLNEED, MADV_POPULATE, MADV_POPULATE_READ, mlock)? (Hypothesis: these options will make it more likely that data is pre-cached and thus fall into fast path more often, but without a guarantee.)

That probably should have been the first thing to try. Too mad the mmap2 crate does not expose this.

Also looking at the mmap2 crate, it chooses some rather opinionated defaults depending on which function you actually call, and it makes accessing things like HUGEPAGE maps somewhat difficult.. and for whatever reason includes the MMAP_STACK flag when you call through this path.

I feel like a lot of rust authors put faith in crates that, upon inspection, are generally poorly designed and do not expose the underlying interface properly. It's a bad crutch for the language.

WIth mmap you have to be prepared to handle unexpected page fault errors due to corrupted volume: Unlike standard read/write, where one can handle the issue, now it can happen anywhere the memory is mapped - your code, third party library, etc.

It gets even unwieldy, and now you have to add additional tracking where access is to be expected. Blindly delegating mmap area to any code path that does not have such handling, and you would have to deal with these failures.

Maybe that's not the case on Linux/OSX/BSD, but definitely is on Windows where you would have it. Also in C/C++ land you have to handle this using SEH - e.g. `__try/__except` - standard C++ handling won't cut it (I guess in other systems these would be through some signals (?)).

In any case, it might seem like an easy path to achieve glory, yet riddled with complications.

While the general point the article is making is correct there are some issues.

- (minor issue) async example is artificially limited to 1 thread (the article states that). The issue is comparing 8 OS threads no async to 1 thread async is fundamentally not very useful as long as you didn't pin all threads to the same physical core.. So in general you should compare something async with num_cpus threads vs. num_cpus*X OS threads. Through this wouldn't have been that useful in this example without pinning the tokio async threads to CPUs to forcefully highlight the page issue, and doing it is bothersome so I wouldn't have done so either.

- (bigger issue) The singled thread async "traditional IO" example is NOT single threaded. Async _file_ IO anything between not a thing or very bad in most OSes hence most async engines including tokio do file IO in worker threads. This means the "single threaded" conventional IO async example is running 8 threads for reading IO and one to "touch the buffer" (i.e. do hardly anything).

To be clear the single threaded not being single threaded issue isn't discrediting the article, the benchmarks still show the problem it's that the 8 threaded conventional and 1 threaded async conventional are accidentally basically both 8 thraded.

> This is thus a worst case, the impact on real code is unlikely to be quite this severe!

I think the actual worst-case would be to read the pages in a (pseudo-)random order.

I always thought that one of the use cases of memory mapping was to improve multiprocessing workloads, where a group of processes don't have to duplicate the same region of a working set. In that sense, maybe it's not surprising that single-threaded concurrency can't leverage all of the benefits of memory mapping.

> One possible implementation might be to literally have the operating system allocate a chunk of physical memory and load the file into it, byte by byte, right when mmap is called… but this is slow, and defeats half the magic of memory mapped IO: manipulating files without having to pull them into memory

This doesn't defeat the purpose necessarily. How about for example, implementing a text editor: I want the best performance by loading the existing file initially (say it is <1MB), and the convenience and robustness of any writes to this memory being efficiently written to disk.

I used to really like mmap for a wide range of uses (having noticed its performance in the BLAST DNA/protein search command) but over time I've come to consider it a true expert tool with deep subtlety, like a palantir.

While author said that C's mmap suffers the same issue, I would argue C's mmap is fine, because C doesn't have async. The issue arises from the mmap crate not having an async read and the confusion around how does async work.

Function calls are also blocking IO then because executables and libraries are mmapped.

No secret. Reading from memory is synchronous and always has been, at least in a normal computer. (Sometimes I think of how you could fit a fancy memory controller in a transport triggered architecture but that’s something different)

See also the classic "Are You Sure You Want to Use MMAP in Your Database Management System?" which mentions this common pitfall of mmap, and others, in the context of DBMS.

[0] https://db.cs.cmu.edu/papers/2022/cidr2022-p13-crotty.pdf

- register

- shadow register

- L1

- L2

- L3

- RAM

- GPU/SPU/RSP

- SSD

- Network

- HDD

The line is drawn depending on what you are doing and how.

Edit: moved Network above HDD. :-)

water is wet