'Maybe the problem is you' ... Linus Torvalds wades into Linux kernel Rust driver drama
'Maybe the problem is you' ... Linus Torvalds wades into Linux kernel Rust driver drama
Open source project chief hits out at 'social media brigading'
https://lore.kernel.org/lkml/[email protected]/
Here’s the source thread.
Tldr: someone wants to put rust in the dma part of the kernel (the part that accesses memory directly)(it’s a memory allocator abstraction layer written in rust which rust code can use directly instead of dealing with the c allocator abstraction layer), is told that rust should use the extant methods to talk to the c dma interface, replies that doing so would make rust programs that talk to dma require some more code, gets told “that’s fine. We can’t do a split codebase”. The two parties work towards some resolution, then hector martin comes in and acts like jerk and gets told to fuck off by Linus.
Martin is no lennart poettering but I don’t try to see things from his perspective anymore.
It’s worth noting that Linus’ “approval” of rust in the kernel isn’t generally seen as a blanket endorsement, but a willingness to see how it might go and rust people have been generally trying to jam their code everywhere using methods that rival the cia simple field sabotage manual.
I don’t think it’s on purpose (except for maybe Martin) but a byproduct of the kernel maintainers moving slowly but surely and the rust developers moving much faster and some seeing the solution to that slow movement as jamming their foot in the door and wedging it open.
Except you're wrong about them wanting to put Rust code in the DMA subtree. As per the article linked below by M1ch431:
In a message to the Linux kernel mailing list, Hellwig wrote: "No Rust code in kernel/dma, please." For what it's worth, the patch added code to the rust/kernel portion of the Linux source tree, not kernel/dma, as far as we can tell.
All they were doing is adding an abstraction layer, within the already existing Rust code, so that rust drivers could communicate with the C DMA code in a uniform and predictable manner. It would have put far more work on maintainers, both C and Rust alike, to have each and every driver implement its own abstraction to the DMA API. Issues would have been/will be filed against the kernel/dma subtree in error due to issues with these myriad abstraction layers.
It’s a duplication of functionality in kernel/dma.
That’s why the submitter didn’t say “I didn’t submit to kernel/dma, checkmate libs!”.
The intent is to duplicate functionality in kernel/dma then get it included directly or linked to.
That’s what the r4l project is trying to do explicitly!
Before you say that kernel/dma didn’t have functional easy to use rust bindings, so the commit couldn’t have duplicated functionality: someone on kernel/dma said they didn’t want that and suggested using the c bindings instead which is what every other language has to do. Which means there was already a solution that was functional.
It’s like if there’s a community bicycle and you bring your drill and tap set so you can mount your bottle caddy and the community says “please don’t make a hole we have to tig in. Just use a pipe strap.” The right answer isn’t to start building a whole new down tube you can tap for an m5 for your bottle caddy, it’s to just use a pipe strap for your bottle caddy.
I didn’t read the linked article (or any linked article about this) because I’ve been reading the mailing list. Reporting on the kernel and people’s behavior on the list is tiring and often includes a bunch of baseless speculation.
To be fair, I'm not sure how "I will do everything in my power to oppose this" is the anti-Rust side "work[ing] towards some resolution"...
That’s tame for the kernel mailing list lol.
The context is that hellwig doesn’t want another maintainer or deal with a split codebase in the dma subsystem which I honestly agree with.
If I were a maintainer in that position I’d be barring the doors too. It’s not a driver for some esoteric realtek wireless card or something.
Even if I didn’t agree with that position it’s normal to only post on the kernel mailing list about shit you actually care deeply about because it’s public and aside from all your fellow devs taking the time to read what you wrote, psychotic nerds like myself watch it and will try to read the tea leaves too!
https://lore.kernel.org/lkml/[email protected]/
General idea seems to be "keep your glue outside of core subsystems", not "do not create cross-language glue, I will do everything in my power to oppose this".
trying to jam their code everywhere using methods that rival the cia simple field sabotage manual.
I am aware of the manual, but I fail to see how adding to a codebase is "sabotage" if it's all generally seen as fine by the project lead - it's far from a hostile takeover.
Would a CIA saboteur even want memory safety as a rule? Just speculating, but I'd say that's unlikely.
Edit: I changed the order of the sentences, as it was not intentionally ordered, and slightly clarified my second thought.
I don’t think the ends are those of the cia, and I didn’t say that the means were either, only that they were similar to those in a famous mid century guide for those trying to halt or hijack organizations.
I don’t think the rust devs are a cia opp, before you ask. I think some rust devs and even proponents of rust who only cheer from the sidelines are sometimes behaving in ways that raise red flags. I think it’s natural and laudable that the existing devs and maintainers are alarmed by that same behavior. It’s their job.
I also think Linus position on rust has been stretched to the point of breaking and I personally find it hard to take positions seriously that distill the complex process of integrating new languages into a very old very large codebase with many full time developers into “Linus said I could”.
"Follow rules to a letter" kind of sabotage manual.
Shit they triggered the raid boss early
OK, who pulled?
Why am I even asking, it was the hunter of course.
LEEEEROOOOOY JENKINS!
FTA: "However, I will say that the social media brigading just makes me not want to have anything at all to do with your approach.
"Because if we have issues in the kernel development model, then social media sure as hell isn't the solution. The same way it sure as hell wasn't the solution to politics.
"Technical patches and discussions matter. Social media brigading - no thank you." -Linus
Yeah, I have to issue an unqualified agreement here. Linus isn't saying no to Rust, he's smackin' that ass for bringing drama out into social media instead of working through it in normal technical discussion channels.
It sounds like he tried that, and nobody with authority responded until he went outside the list. Even now, Linus hasn't actually answered the question of whether more rust code should be allowed.
No offense, but reading through the comments it's apparent you're not very familiar with systems programming nor linux development. This is a common problem with vocal 'rustaceans', rust is their hammer regardless of the domain.
Although considering rust is prudent, there are still a ton of advantages to using C for systems programming. It is not a binary choice, there are pros and cons, and every project should choose what aligns with their priorities.
No one has ever stated that linux will be in the kernel. It was 'go ahead and give it a shot', which includes convincing maintainers to accept your patches. Linus has delegated trust to subsystems maintainers and an established process.
Hellwig could have been more tactful, but like it or not, arguments against a cross-language codebase have merit. Framing it as a 'clear confession of sabotage of the r4l project', attempting to weaponize the CoC, and trying to drum up an army via social media was all out of line.
Success was never a given, if they want r4l to succeed then they have to get patches approved and crying wolf ain't gonna cut it.
This whole thing reads not like a codebase versus, but a traditional engineering approach (don't act like you can patch this once you release it - get it done so it's stable the first time) versus the more modern "move fast and break things" approach.
I am so glad Linus just came out and said it. I was pretty upset at Hector too in the other thread the other day, and I especially didn't appreciate a call to remove a major developer from the kernel because Hector wasn't getting his way. Very militant action on Hector's part where it just wasn't necessary.
Hector, if you're reading this, communication skills are just as if not more important than your Rust development skills, and frankly your communication skills lack.
You seem to be in the loops of the linux kernel?
If so, ive known hector from way before when we was part of f0f, or TT as they were known before, doing wii homebrew work.
What you describe is what my experience was with him 14 years ago too. The guy is smart, he has a very good skill set and knowledge, but his communication skills were lacking back then too.
Granted, both he and myself were still teenagers and students and we were wild, but i had always assumed he grew up a bit since then...What you said is spot on, and i hope he does read both of these. And if he does :
Marcan, you might not know who i am anymore, but ffs man. Dont screw up your love for all of these by keep kicking the hornets nests. You did it with devkitpro, emudevs when the nier news dropped and with rossman too. Stop it, its for your own good.and with rossman too.
I decided to read replies: wierd, they suggest accusation is overblown.
I decided to read context: WTF is this?! Unholy shit, dear Faust, what did I read? What a deflection!
I thought I was terminally online with mental disorders, but this makes me look most grass-touching and sanest person.
I can understand their frustration, having multiple other rust for Linux project maintainers quit over nontechnical rust aversion.
And Linus continues to (democratically?) avoid the subject with this response.
As a rust for Linux volunteer you have to be incredibly demoralized reading this mess almost every other month.
Part of why linux has been a successful long term project is by making decisions conservatively. Other projects like cURL do the same. Incremental improvements over time.
It seems like there is a culture clash with the rust devs who are pushing for changes faster than the long term project maintainers are comfortable with.
@semperverus Just from the small interactions I had with Hector on mastodon I can see he gets very unreasonable about small things and does not accept the possibility that he may be wrong, despite evidence. So leaving linux and mastodon because of rust is totally on brand for him.
Hector has a rap sheet of crash outs and drama.
So now we've lost a very good developer, and the question of rust in the kernel remains unresolved. This is the worst possible outcome.
Part of being a good developer is the "working well with other human beings" part. Linus himself took a hiatus to improve himself in this area.
Another part of being a good developer is to work within and adapting to the frameworks of an existing project, especially if you are joining at a later point. In this context, it would be the R4L folks joining the project known as "the Linux kernel."
Hector failed on both counts. He has programming skills, but that's not all that's required.
The times when a single developer was important to Linux were in the 90s.
This is just a dick measuring contest.
I'm relatively new to Linux and the FOSS scene, but I'm not sure how I feel about the unquestioning devotion to a single person. It seems antithetical to the entire philosophy.
Even if he was maybe right this time...
The dude did a complete 180 as soon as they heard from Linus, like daddy made his decision, and it's final, or some shit...
Edit: To be clear, I understand why developers respect and listen to Linus... I just think there are fundamental issues with this kind of top-down management of such a colossal project, and the desire to defer to one person seems antithetical to the FOSS philosophy.
I don't think it's blind devotion - most of us would acknowledge the guy can be a bit of a dick sometimes.
But we're also grateful. Without his silly idea in the 90s, linux wouldn't exist. Computing today would be massively different - big, commercial, massively expensive unixes like Sco and Solaris dominating the industry. My main hobby for 20 years would be very different. My career for six years wouldn't exist.
That Linus has stayed an actively contributing member whilst not selling out in any way at all for 34 years is... wow. Could you do it? I'm certain i couldn't. I have neither the ethical strength nor moral compass to do it. And I'm certain if he dropped out, some of the massive egos that satellite around Linux, or the monetizing businesses would seek to take over and twist it to their needs.
And, y'know, on the matter of technical detail like this. He's nearly always right. Seriously, look it up. He's not polite, he's not diplomatic, but he's nearly always right. And when he's not, he'll admit it. Again, not your normal human.
So yeah, that's why we respect him and, when he talks, we listen. Even if it's not something we're involved with, it's usually an interesting ride.
I don't know if it's complete devotion, but Linus has a reputation he's earned the difficult/hard way.
If he says something, people should take it seriously and consider his words. That's not to say he's right all the time, but you'd better have a damn good reason for disagreeing with him.
Also a reply so you can understand a bit how things typically work in FOSS projects.
There's a democracy in healthy ones, but ultimately, there has to be someone at the top that has the final say. The project maintainer/main contributor. Someone who gets to be the tie breaker, or absolutely final authority on what does or doesn't make it into a patch/version/etc.
This is extremely common, and generally healthy, in these kinds of ecosystems.
If Linus genuinely went off the rails, the kernel would just get forked. Even right now, if the way the mainline project is run doesn't work for someone or what they are doing, that can and does happen.
Linus has power because the people who contribute to the project allow it, and they allow it because over the years he has consistently endeavoured to make decisions based on what is in the best interest of the project. People want him in charge, because he has done, and keeps doing, a really good job.
He hasn't always been nice to deal with, and he can get spicy when he puts his foot down, but whem he does, its not on a whim. And if he's wrong, and you can articulate why and how, in good faith, he won't ignore the logic of what you are saying out of some childish sense of pride.
Unless a major corporate sponsor of Linux forks it s fork is going no where. The amount of company work that goes into the kernel is massive.
Its not antithetical to the Foss philosophy. Thus happens because Linus is a trusted figure, something he's absolutely earned. He didn't just buy control of some product, or get promoted to this position by a company. Many great open source projects have a BDFL. If people lose their trust in the projects BDFL, they fork the project.
Also, the kernel is really just one part of Linux. Distros include a whole bunch of software they choose to deliver a full OS (hence the Gnu+Linux people). Linus doesn't have control over the OS as a whole, just the kernel.
Edit: Just finished reading the chain, what do you mean the dude did a 180? He expressed frustration that Linux only criticized him, further criticized the issues with the kernel development process, and said he was giving up being part of the kernel.
It’s just respect
It's not unfounded and I don't know of a time when Linus wasn't right in the end. But I wouldn't say it's blind devotion he would be turned on in a second he betrayed his principles. Also FOSS is not about lack of ownership its about sharing code for the greater good. Every owner of a project knows that a project can be forked in an instant.
It's not devotion, people can fork the Linux kernel if they want. He's just the one in charge of the mainstream kernel
Honestly I kinda wish the Rust devs would rather go and support a project like Redox OS and then maybe we can have less drama about all this.
Tamest Torvalds quote I've ever seen so far.
that article is horrible to read! every paragraph starts with quotation, but then never closes it😵
Could someone explain the backpedalling if rust is more secure ?
it's a different technology and paradigm that the old guard would have to take considerable time to learn to be as productive as they are in C. it requires a different way of thinking about systems.
basically the rust-in-kernel-gang includes none of the "main" kernel team because they are busy building the kernel. this is an experiment to see if a second programming language can be successfully integrated into the kernel at all. if they try to force their way in, that's going to cause problems for everyone.
My gut tells me that any benefits of adding Rust is massively negated by the addition of a second language.
If one wants to write Rust, there is always Redox and probably a bunch of other kernels.
I like Rust, but it's for sure an over hyped language. In a year or two, people will push for Zig, Mojo or some new pure and polished functional low level language. Maybe a Scheme or a Lisp? That seems to be what the cool kids use nowadays.
Or maybe we'll just replace the kernel with an AI that generates machine code according with what should be your intention.
C'mon man, this is just a textbook fallacious slippery slope argument. Rust isn't some brand new language whose stable release was less than a year ago, it's over a decade old now. Scheme and Lisp are interpreted languages for God's sake, it's borderline* impossible to use them for kernel programming.
Also I'm pretty sure the whole point of the Rust project that all this drama is centered around is to keep Rust code separate from the kernel. From what I understand the whole point is to maintain Rust bindings to the kernel API as a separate project, so that if developers want to write a driver in Rust, they can without having to rewrite those bindings themselves. But the kernel code itself will still be all C code. Now I'm not a kernel developer, and the last time I wrote a driver was for my operating systems class in university over a decade ago, so take that with a grain of salt.
* I say borderline because anything is possible with code if you're creative enough, but anyone trying to submit Scheme or Lisp code to the Linux kernel is gonna get laughed off the Internet
I don't think you get my point.
Of course I don't mean that you should introduce Lisp or Scheme into the Linux kernel. However, I don't rule out anything when it comes to the future of programming. Kernel programming isn't that special. If you need to make a scheduler, dynamic memory manager or an interpreter, as part of the kernel, because it solves your problem, you do it. Maybe you want the kernel to generate thread optimised FPGA and micro code on the fly? And this is done with some kind of interpreter. Who knows.
My point is that it's probably a bad idea introduce any new language into the kernel. A new backwards compatible version of memory safe c might be a good idea though. If it can be done.
Haven't touched the Linux kernel in 10+ years, but my guess is that a good approach is to write a new micro kernel in Rust. One that is compatible with most existing drivers and board support packages. And of course it has to maintain the userspace ABI and POSIX yada yada. Probably what the Redox project aims for, but I don't know.
Keeping the Rust bindings in a separate project might be unnecessary though. I'm sceptic about allowing upstream drivers written in Rust just because I find that there is such a great value in sticking to one language. I also know that many kernel developers are getting old and it gets harder to learn new languages the older you get. Especially if the language comes with a decent share of sugar and bling (the minimalism of lisp and c is valuable).
If there is a problem finding driver developers that want to write C code, then sure. But breaking the flow of the senior maintainers/developers likely isn't worth it. Unless they ask for it.
And also, I really haven't been following this Rust in the Linux kernel debate.
Technically there's no such thing as a compiled or interpreted language. Python compilers exist. You could write a Scheme or Lisp compiler. Racket is a Scheme that can be interpreted or compiled
Can someone distill the good faith argument against rust? Is there one?
Can someone distill the good faith argument against rust? Is there one?
The problem is that even if it's objectively better, you can't magically convert everything instantaneously, and it's a lot more work maintaining rust and C versions of the same code until everything is re-implemented in rust.
Only one compiler nailed to LLVM. And other reasons already mentioned.
it's more niche than C, has less competency available, works very differently to C, and requires a whole new toolchain to be added to the already massive kernel compilation process. for it to be plain sailing adding it to the kernel some of the worlds' foremost domain experts on operating systems would have to re-learn basically everything.
also since rust is just coming up on 15 years of existence without a 1.0 release, there's no way to ensure that the code written today will be considered well-formed by the time 1.0 hits.??? Rust 1.0 was released 10 years ago and since then there have been no breaking changes.
Syntax ugly
Rust people are so annoying.
counterpoint: stonewalling c programmers are so annoying.
he looks like some kind of dino, i cant figure out which one