Rust in Linux Revisited

I don't see why it's so important to rewrite parts of the kernel in rust, or why Drew is so sympathetic to the cause. The linux project uses static analysis tools which provide some of what rust guarantees (outside of unsafe).

The language isn't the problem, it's the architecture. In Linux, all of the drivers run with the rest of the kernel. A bad driver can take down the whole thing, or perform arbitrary badness.

In Drew's own Helios microkernel, the architecture is right, and it's ready to have folks start piling on drivers in user-space. Those drivers can be written in pretty much any low level language that can produce ELF binaries.

It's surely a better path forward to slap a Linux syscall API on top of Helios + some new drivers than to clone the whole linux system architecture in another programming language. At least then, we would have gained something tangible (drivers can't ruin the kernel). Instead of having another Linux written in a different language, with all the same design flaws.

> Here’s the pitch: a motivated group of talented Rust OS developers could build a Linux-compatible kernel, from scratch, very quickly, with no need to engage in LKML politics.

The BSDs, Solaris/Illumos, and Windows all have tried, some more than once even. I think there have been at least 4 Linux kernel ABI compatibility layers for those OSes. They've all failed. And all future attempts will fail too for the same reason:

  The Linux kernel ABI is not
  specified anywhere, and it's
  insanely large (and growing).

Besides the system calls (easy), many ioctls (hmmm), proc(4) (oof), and many ioctl-like ABIs within ABIs, there's loads of driver-specific ABIs and things like security modules and what not that add their own ABIs and which you'll invariably end up having to support. The last effort in Illumos land foundered on proc(4), IIRC.

To top it all off that undefined (but stable) ABI is a moving target: it's always evolving / getting extended.

The only thing you have to help you is that whatever the ABI, Linux [mostly] commits to maintaining backwards compatibility with it. If your compatibility ever gets good enough, that will help you, but in the meantime you'll be chasing a never ending and ever growing ill-defined ABI.

Good luck with that!

I am split on this. Maybe I am idealistic and naive, but I will always wish people can just stop fighting, and work together. Fragmenting the community and starting a project out of grudge, is always the last resort, IMO.

OTOH, I also recognize sometimes it is really that bad and going your own way is the best thing to do. But I think Linux isn't at that position yet.

Duplicate of https://news.ycombinator.com/item?id=41400493

> Here’s the pitch: a motivated group of talented Rust OS developers could build a Linux-compatible kernel, from scratch, very quickly, with no need to engage in LKML politics.

Rust is many things, but devoid of political drama it is not.

Someone in a sub-thread accused Drew of being incurious, and reading the article, I kind of agree.

It's a very polite, high-effort, superficially humble piece of writing, that nevertheless boils down to "You guys should probably leave us alone and work on stuff we don't need to worry about".

Now, working on a Linux fork as a "proof of value" thing could be interesting, but it also means that this hypothetical Linust project would be stuck forever chasing Linux's API decisions without any power to influence them (and, if recent history is any indication, quite a lot of hostility from OG Linux maintainers).

I can't help but notice that Drew's plan doesn't include any exit strategy, any point where the projects merge or Linux starts taking components from Linust or something.

Maybe Drew thinks that forever being stuck between shadowing a concurrent project's API and trying to convince its billion users to switch to yours instead is an attractive prospect. If I was a Rust-on-Linux developer, I'd find that patronizing.

There's an attempt at heading off the driver commentary with a note at the bottom. The note makes some hand-wavy thing which is essentially "assuming you have an army of people with specifications, this won't be a problem", but that assumption is crazy. Fuchsia is a real OS effort funded by Google (at varying levels of funding through it's lifecycle, which is still ongoing AIUI) - I worked on Fuchsia for a number of years, from approximately the time the kernel had become usable, through to launching on nest hubs where it replaced the existing OS from bootloader to GUI. Drivers remain a huge problem for Fuchsia despite funding. SOC vendors don't provide docs or assistance, even for serious efforts, they demand money first - they're literally holding the software world hostage with this activity.

After leaving Fuchsia and reflecting on the whole thing, I really believe that despite the many problems Linux may have over the long term, the only viable strategies for a broadly and practically usable safer OSS system are either: someone donates at least $2bn to get it done in a PBC well managed and focused over a decade, or you slowly mutate Linux into that thing. Anything drastically different is, for now, dependent on the world being a different shape / people & companies having different attitudes than they do. I'd love to see something like Redox prove me wrong, but real world experience suggests that's extremely unlikely. RISCv is the one thing in flight that might "change the shape of the world" enough to alter this, but RISCv comes with it's own stack of long challenges to overcome to meet the originally stated goal of broadly and practically usable.

Writing a core kernel and usable userspace isn't something to be trivialized, but by cost and work volume, getting a broad set of drivers is far far more effort by an astronomical margin. The OSDev community has thousands of OSes to explore, and barely any drivers that do things a user actually cares about.

> Two years ago, seeing the Rust-for-Linux project starting to get the ball rolling, I wrote “Does Rust belong in the Linux kernel?”, penning a conclusion consistent with Betteridge’s law of headlines. Two years on we have a lot of experience to draw on to see how Rust-for-Linux is actually playing out, and I’d like to renew my thoughts with some hindsight – and more compassion. If you’re one of the Rust-for-Linux participants burned out or burning out on this project, I want to help. Burnout sucks – I’ve been there.

Now this compassionate revisit offers the same conclusion: don’t do Rust in Linux.

This person read that email about one Rust Kernel developer resigning because of burnout. Now he goes into how the Linux project is a “burnout machine” and how his heart goes out to the “developers who have been burned” (how did we get to plural?).[1]

The “so where do we go now?” almost gets ahead of itself before it says in the next paragraph that “the path is theirs to choose”. Well yeah because the only person who implied there was a crossroads is the author here.

The predictable conclusion is to abandon the project and do something adjacent to the Linux Kernel. But what if you cared about working on the Linux Kernel specifically? What if you cared about the code in the Linux Kernel itself, its long term health… hush, hush now. You are burned out and don’t know what you are saying.

The penultimate paragraph then declares that the Rust-for-Linux project itself is “burned out” (“and that’s awful”).

Who needs enemies with compassionate friends like this.

[1] How often do we read about maintainers suffering burnout? Every day? Do we then declare that the project is a failure, even when there are other maintainers left on the project?

I must be missing something obvious. I’m not aware of any too-frequent toxicity in the rust lkml.

Just seems like a lot of groundwork is still being laid, without much even having been started.

The article makes it seem like things are actually being rewritten in rust.

Are they?

The reddit threads on this subject were interesting, with hundreds of comments in r/programming and r/linux all bemoaning the lack of Rust adoption in the kernel and the supposed mistreatment of the Rust maintainers.

Are these people just users with a strong opinion about what language their kernel uses? If they're Rust devs and reflective of how popular Rust is in general, where are all of the killer apps written in Rust? Why do Rust programmers seem to mostly just produce partial, feature-incomplete rewrites of Unix utilities?

with respect to linux kernel rust projects: is there anything new that is complete and ready to use or anything that existed in c that has been rewritten that is stable and has reached feature parity with the existing implementations yet?

Cygwin is a compatibility layer between linux syscalls and windows. And that is all userspace. So if you somehow implement everything in there plus the raw kernel functions, maybe there is a chance I think.

Could not agree more.

I think there’s a parable in the Bible about this one:

No one puts new wine into old wineskins; or else the new wine will burst the wineskins and be spilled, and the wineskins will be ruined. But new wine must be put into new wineskins, and both are preserved.

He was specifically talking about bringing new paradigms to an old system. Sometimes, it’s just easier to start over. It has been my experience in programming that starting over almost always yields better results, but it does take time.

That’s why apples m1 was so successful. They took the time to rebuild just for apple. But it was 100% worth it.

The counter has started. I am looking forward that drop-in kernel, that could replace Linux on all my production systems "has-is".

This sounds very easy. Very.

This is an interesting take. Has DeVault done a lot of Linux kernel development?

TLDR: In response to the recent resignation of Rust-for-Linux's Wedson Almeida Filho [0] and consequent debate over the politics of Rust in the kernel, Drew DeVault suggests that the Rust for Linux crowd build their own ABI-compatible Linux clone, reasoning that a clone is easier than a greenfield research OS (like the existing Redux).

It's not a bad idea, imho. Rust folks love rewriting things, and are often quite good at it. It would keep both the curmudgeonly kernel Never-Rusters and the Rust evangelism brigades happy (by keeping them apart). And it's a great proving ground for Rust's claims - assuming licence compatibility, successful rewrites of individual components could be lifted by Linux down the line.

[0] https://lwn.net/Articles/987635/

I expect the result of Filho's tantrum will be the ouster of Ted Tso from the Linux kernel.

I think he is wrong on two accounts:

1. While the Linux guy mentioned here is not a great politician and his rant was pathetic, his technical argument was sane. You can put months into creating the perfect data model for your Rust implementation but Linux have a habit of completely changing things every year or so. That would make Rust the bottleneck of Linux development

2. These "hugely talanted" Rust devs are competing with ten times more talented Linux devs on their home turf.

My point is, stop creating drama. This task requires hard work and compromises.

These arguments are pointless because in a few years you can have your AI of choice translate Linux to your language of choice.

The real question is will you even bother to do it? Will it matter at that point?

It's more or less working for ladybird browser, could work for a kernel.