FreeBSD-rustdate, a reimplementation of FreeBSD-update

Thank you, thank you. I haven't reviewed this, and I may or may not use it, but wow was freebsd-update from 13.0 to 14.0 slow, especially since I run my freebsd systems off spinning drives, and most of them have src.

I did some looking at freebsd-update, and it's quite the shell script, which makes it daunting to consider changing. All the forking and whatever doesn't seem great; using data structures should be a lot simpler!

But my systems running freebsd-update spend an awful lot of time on I/O, and not so much time on cpu, as I recall. I think the general flow for a replaced file is gunzip to the update dir, then install to the destination. This means all the files are written twice, and I thought there was a third write that I can't remember at the moment. Install doesn't have an option to decompress, but it'd be super handy if it did --- then you could write the file only once to the destination directory and move it into place.

I'm never a fan of quite as many files in a directory as freebsd-update gets up to either. Given that everything is already named by hashes, it's not too hard to have 16 or 256 directories named 0-f or 00-ff that get a slice of the files. Large directories have gotten better over time, but they're still slow.

There's some ability to exploit the embarassing parallelism available too, but that's hard in a complex shell script, so some other environment makes sense. I didn't look, but hopefully that's a knob in rustdate ... Sometimes you'd want it and other times one thing at a time is better, even when it's slow.

One feature I know update can do that I want to look at more but haven't had time is the zfs boot environment stuff --- seems like with the right arguments you could do the update on an inactive environment and reboot to switch, instead of the default way which does the install on the live environment but you could reboot to switch back. I'd be much more patient if the live environment continued working and I could reboot at my convenience once the process finished. Especially if I might be able to run pkg upgrade in the boot environment before the reboot. If rustdate supports that too, that'd be neat.

> This is freebsd-rustdate, a reimplementation of freebsd-update.

> In usage, it’s expected to be similar, but not identical to freebsd-update.

Doesn't reimplementing it to not be identical, defeat the point of reimplementing it?

(since it can't be used as a drop-in replacement)

See https://rustdate.over-yonder.net/speed.html for the numbers.

Would you mind mirroring to a repo, with a license, so others might view?

Suggestion: call this something else - because 'rustdate' makes me think 'rust date', which makes me believe this is date/time related.

Given that sh can print everything as it runs, it would be nice to know what is it that takes so long, instead of doing a full rewrite.

I remember once replacing a filter written in pure bash by an awk one, the slowdown was around 2 minutes v awk which was a second.

Given the official stance of moving to pkg-base, updating freebsd-update with an incompatible rewrite in Rust seems like yak shaving.