Generators are dead, long live coroutines, generators are back | Inside Rust Blog
Generators are dead, long live coroutines, generators are back | Inside Rust Blog
Want to follow along with Rust development? Curious how you might get involved? Take a look!
What is this title
You're viewing a single thread.
Guys and girls, This problem has been solved in 1980 already. Why are we still struggling?
It's called Communicational Sequential Routines. Like, why do we have to come to full circles and Rust after a decade still hasn't solved the "Async"/Multithreading problem. It's mind boggling to me, Im sorry, people are gonna hate me for me being frustrated, but I just can't say that it's irritating that Rust, a language that really has offered many great ideas, STILL hasn't figured out how to properly do things like channels and sequential communication? A concept that has been invented since the 80's? Or if that doesn't fit at least see how it could fit Rust's standards?
It's HORRIBLE that Rust as a language and community STILL uses Tokio and Async-Std and we can't even decide always which one of the two to use and while they also have some incompatibilities afaik, and this is just for ASYNC. Multithreading and Async? God bless your soul if you're going to do that in Rust, because prepare for debugging for days and for many weird issues and caveats.
Go, a much MORE inferior in features language has done it. What makes it so difficult to inherent such or a similar approach? It's a simple concept. Channels. In and out. And syntax and scheduler like Goroutines to make it easy to have async AND multithreading without the caveats Rust has nowadays if you were going to do such thing.
First, this post has enough misconceptions about the state of async Rust that I don't think you've used it much, or at least not recently. Second, it demonstrates a huge misunderstanding of how Rust works vs Go that I don't think you appreciate why the feature set is different.
To answer the first, tokio is compatible with Rust's standard async support, and there are alternative runtimes to tokio as well that are also compatible with std. I'm currently working on an async-heavy project, and tokio is an implementation detail and only imported when initializing the runtime loop, most of it is just standard async/await syntax. So the user doesn't need to care most of the time what async engine is running.
Tokio and other runners can be configured to be single or multithreaded, the API is designed such that it doesn't matter which you pick. I run it multi-threaded, and I use channels as well to communicate across async contexts. My code is structured very similarly to how I'd structure it in Go, it just doesn't have the same, clean syntax.
What makes it so difficult
A few things:
- Go has a massive runtime which handles everything from memory management to scheduling
- Go has very limited memory safety guarantees, and nothing to prevent you from shooting yourself in the foot with concurrent memory access
- Go was designed around goroutines and channels as first class features, Rust adds it as an option (compare regex in perl vs other languages, lists in lisp vs other languages, etc)
The first two are the most important here.
Rust has no runtime, so it can't and won't do things like provide a default async runtime or scheduler implementation. This means you'll always need to configure and start your async engine, which means it needs to be modular. It turns out Rust doesn't need a default implementation to help libraries abstract over async, it just needs a handful of traits. This also means standard libraries won't plug in to the scheduler (e.g. when a socket or file read blocks in Go, it yields to the scheduler, whereas in Rust it just blocks), so you'll need an async-aware alternative if you want to avoid blocking. With Rust, you don't pay a penalty if you don't want async, but you do in Go because it's baked in to the scheduler, standard library, and language syntax.
One final quip about Go since I have used and loved it since day 1 (convinced boss to switch once 1.0 landed and we didn't look back). For all of the features Go provides over async, there are still a lot of surprises. For example, why is
mapnot threadsafe? The standard library bakes in async, why not makemapatomic? Slices have a mostly safe interface, but they also do surprising things (e.g. you can copy over its backing array outside the bounds of the slice), and they could easily be made atomic and thus (mostly) async-safe. So Go gives you a lot of built-in features, but doesn't go far enough to make it actually safe.And that brings me to my second point: memory safety. Rust won't let you do stupid things like write to a Vec or HashMap from multiple parallel execution contexts, whereas Go will. So while you give up certain syntax niceties, you preserve memory safety, so your code in Rust will always be memory safe even if you get lost in the syntax (which isn't that bad imo). And since everything is modular, you can replace parts as needed if you need to hit certain performance targets or something.
You can still use channels and spawn coroutines, the syntax just isn't built in. And that's my third point, Rust isn't designed around any given use case, whereas Go is. So if syntax matters more than the benefits Rust provides, use Go. But if correctness is more important and you're willing to put up with some verbosity, use Rust.
Bro, it's not a battle. Like Golang Vs Rust. You treat it like a battle. "Go has very limited safety guarantees in comparison to Rust", "Go was designd around go routines, Rust adds it as an option" as if this is an advantage? How exactly?
In any case, just look at Primeagen, any Rust programmer. Or do your own websocket server in Rust vs Go.
Go = 10 minutes Rust = 10-20 hours.
Does that mean that Rust is worse? NO. Absolutely No.
You though make it seem like a hecking war like Go Vs Rust. Jesus chill bois. I know Rustacians are fanatics but chill.
Go provides you a type-safe way and a clean-very-self-documentable, efficient, multithreaded/async/parallel way (yes yes ... I know ... yes ... they're all different, don't be nitpicky jesus christ, I know somebody in the commends will try to "correct me"), to write programs VERY fast in a as type safe way as a company usually needs.
I'm not paid by the hour, I'm paid by the month. I'm not gonna spend 3 months just dealing with Tokio, Features, "async" which is not a feature but an option in Rust which is horrible nowadays, our machines are VERY powerful to be able to do async things very fast, it should have been a feature which you might use it or not like in Go. NOBODY forces you in Go to write a Goroutine, it's just a language feature, get it? Mutlthreading, a hell in comparison with go.
Like, sometimes we just have to accept ... each tool is best for its own job.
You want a VERY performant CLI tool? A system kernel mod or something that has to do with memory so much that Go will just be really bad decision? Go to Rust or C++. EACH TOOL, FOR ITS OWN JOB. It is not a heckin battle.
And sometimes we just have to accept, that no language is flawless. Stop trying to excuse the fact that Rust is very slow to program on and document in comparison with languages like Go. Go is not flawless either, what language is?
Go
- has no pattern matching
- has garbage collection. horrible if you want an app which is very memory heavy and with many arrays, structs and lists.
- Structs - Structs fucking amazing in Rust. I love them to the point I almost said to myself "Im not going back to Golang".
- Enums ... same with structs. They're very powerful
- Generics - Whoever "Go" elitist says "But but ... Go has generics" ... I swear to god, I will pick his nails and throw his body to the ocean. Generics in Go suck. Still. Even 1-2 years after their release (1.18 was it? and we're at 1.21 at go?) Rust has proper generics
Like do you get me? You make it seem like a battle between languages. Stop being fucking elitists. No language is perfect. Rust is SLOW to PROGRAM on. FAST on performance, but just to deal with the borrow checker its like it's programming for you instead of you programming for the compiler and some changes require massive changes. And this steap learning curve, and mistakes, and really big complexity, make Rust just a NONO for small project. Do you really wanna make a websocket server in Rust? Be my guest. In Golang I can make it in ~50-100 lines and it's perfect. But in Golang, when memory performance starts to matter and it's not insignificant, which happens after decades most likely when you already have the budget to refactor, like Discord did (which just burns money of its investors btw it has no profit).
My point being 1 -> STOP being fucking elitists and "RUST IS THE BEST FUCKING LANGUAGE". No its fucking not. No language is. Or trying to excuse the choice of not having Async by default and being so overly complex at times you just require hours to debug some sht.
-
-> EVERY language has its advantages and its disadvantages. CAN you please ... for god's sake ... accept that? Just for once? Rust has big flaws when it comes to async, parallel and multithreading programming. It's a mess. You'll be debugging for hours or days.
-
-> Every language = a tool. Every tool has is used for a specific reason. It seems that Rustaceans fell in love with Rust so much they wanna make Rust everything. Dude or girl, girl or dude, that's just ... bogus and out of mind. It means you're the cult. You're not in a cult, you're the cult when you do that. No language is perfect for anything.
-
Accept judgement. Okay? Languages just suck at somethings. Every - single - one - of them. NodeJS at multithreading, typesafety, documentation, memory (copying everything) and making programmers careless. Go at not having enums, proper generics, pattern matching, garbage collector etc. Rust at multithreading + async + parallel + making simple servers + borrow checker making it a PITA to program and design things the way you want to, instead you design things the way the borrow checker wants and it drains MUCH of your time redesigning things when you make mistakes, youre a human, u will make mistakes. ... C++ and being so overly complicated you can barely read it and it makes stupid people think they're smart because they used the "new C++22 feature" while there was a much simpler and efficient way to do it. But hey, they're smart because "they follow the news and C++ has a new feature they wanna use in production which makes code slower". C for being so simple and barebones and without language tools, like C++ too, it requires things like Ninja etc. to compile the headers etc. or do anything complicated. Thus why it's not used by companies to make simple microservices. hello? It's a PITA to write in languages like C++/C/Rust for a simple micro-service, and insignificant until you are actually a multi-million-hundrends or billion company with much traffic. Like, accept fucking judgement. It's not bad for once. Some languages just such at some things. Even my brother, who works at Cadence, a software company for firmware for Chips and testing chips like ARM for Apple etc. a fortune 500 company, used python because it's just ... more convenient instead of having to recompile C++ every f'in time on each client's CPU and there are countless when you're such company. But Python is really bad at many things as well, but see? It has its own very nice uses despite that my brother works for almost a decade with C++, he chose python for that job because he knew it was the correct choice. Stop trying to excuse everything.
Rust sucks at async & multithreading. Period. In comparison with other languages, it's a PITA to debug and deal with. That's why the main rule in multithreading is "do not use multithreading unless you have to". Well, in Go, you kinda dont have to think about that despite that you have to learn some patterns for data-races safety. I can make you a multithreaded program barely thinking about the multithreaded-side of things because that's the scheduler's job. It was built for it. But it sucks at many other things. Like Rust, just accept that and we can keep going on with our lives
I didn't say it's a war or anything like that, I actually like both languages, I just don't see much of a point to Go anymore for my priorities. Look at my final paragraph:
So if syntax matters more than the benefits Rust provides, use Go. But if correctness is more important and you're willing to put up with some verbosity, use Rust.
Go makes many things easy, and that's absolutely a good trait to have. My point is that, long term, it has enough footguns that I'd rather put in the time to use Rust, which will prevent most of those by forcing me to contend with concurrency issues upfront.
If I want something quick, I'll probably use Python. With Python, I have a lot fewer options to screw myself over with concurrency long term, which means worse performance, but I don't need that for something quick.
I think Go makes sense if your project will remain small and focused, performance is really important (but not critical), and you're familiar with it. It's easy to write, easy to deploy, and has a high quality standard library. But long term, it has a lot of footguns that become serious issues as projects get larger.
Generics
Agreed. Go doesn't need generics, what it needs imo are scope guards similar to Rust's
Mutexor Python'swith.deferis close, but it's too easy to make a mistake. I have longed for deterministic destructors, but Go leans way too heavily on its GC to make that feasible.Interfaces are good enough in most cases to provide what generics are intended to provide. I think there are problems in the implementation (e.g.
interface{}((*int)(nil)) != nil; I've run into something similar way too often), but the concept is fine.- I never said Rust is the best language. I just said I don't have a use case anymore for Go, because either I'll want Python or Rust.
- Yes, and I named some both in my original comment and this one.
- Again, I don't use Rust for everything. I made that clear I'm my original comment and this one.
- Yes, Rust's async is a bit tedious, and I mentioned that in my initial comment. However, you also get a lot of help from the compiler to eliminate most tricky concurrency bugs. I think that's a fair price to pay for software that's intended to be maintained long term. Go doesn't, so I can't recommend it for larger projects. Python gets halfway there, so it's good enough for prototyping and smaller projects, so that's what I reach for instead of Go, and that's with using Go for almost 10 years (I started with 1.0).
C++
Yes, I think C++ is actually harmful, I see no value in using it whatsoever. The only reason I think it is still being used is inertia, so many libraries use it that it just has the lowest barrier to getting started. But that's not a very good sales pitch (use C++ because everyone else does!).
I much rather write in C than C++ because at least that way I know I'm not likely to make assumptions that do not hold, so I'm much more careful.
Rust sucks at async and multithreading
That's just not true. It's perhaps the best language I've ever used for async and multithreading, and I've used plenty.
But I seem to be looking for something different than you. I want correctness so I don't have as many tricky concurrency bugs to contend with in production, and I'm fine with development being a bit slower to get that. Async and multithreaded Rust can still have bugs, but at least not concurrent modification bugs.
You seem to be looking for quick prototypes, and Rust won't give you that. That said, Rust has gotten a lot better in terms of async ergonomics in the last year or two, and I've found it incredibly productive for my async-heavy projects that the difference between prototype and production code isn't that large. Once I noticed the kinds of issues that the borrow checker catches, I adjusted how I structure the code to be more provably correct, and I very rarely have compile issues anymore.
Go makes it way too easy to write multithreaded code, to the point where it's easy to assume things are correct when they aren't. Go is technically memory safe, but only in a "we'll protect the system from your mistakes" sense and not a "we'll help you avoid the mistakes" sense. Python is also memory safe, and it comes at the problem by making it difficult to write multithreaded code in the first place. That's absolutely fine for prototyping.
problem with interfaces in Go, is that theyre not compile time. which means
- No errors - debug later (HAHAHAHAHA EVIL) jk
- Runtime-Casting.
So, tbh I dont think interfaces are a solution. Problem is that Go was built with "Simplicity in mind" and "Remove every single feature modern languages have" like enums ... or generics .... which were both made for very good reasons. And I hate that. It makes me think that some kind of hipster made that language (yes I know who the "hipster" is :P damn you Rob Pike and your hipster mind) and now they're trying to retaliate with "putting Generics 10 years later" ... generics that barely work or are useful at all, let alone the things they still don't have but should have. I dont wanna pollute my package namespace with ""enums"" that have HUMONGOUS names just to make them different from other enums. Reminds me of C when Golang is supposed to be ... "the modern C" or "how modern C should have looked like"?? (which is stupid assessment imo, it's not comparable in any way) As they advertised it? Or at least I've heard that many couple of times.
Well let's not be unfair, aside from things like enums or other features, sometimes generics do work, but in comparison with Rust's, C++, C#, Java's generics they're not comparable at all. It's like a drawing on top of Go. It doesn't feel like it's Go-ish and that Go has adopted generics properly at all. But still, good enough for making performant but not critical as you said projects, in an efficient way without having to deal with memory too much and still keeping some middle ground until you need the "critical" part.
tbh happy we came to a middle ground. thanks for discussing through it.
Yeah, no worries.
And I think the idea of interfaces is fine, just not Go's implementation. Interfaces could be a compile-time, duck-typed abstraction with very basic runtime reflection (basically just a type-assertion switch), but they're not. They instead leaned way too hard into it and made interfaces a value type, when they really shouldn't be like that at all.
The vast majority of the time, I just want to pass a concrete type as an abstract type, and occasionally check if that abstract type has any extra features (e.g. does this
io.Readerhave aClose() error?). I've done runtime reflection a few times in Go, and every time it was a disgusting hack.Rust goes the opposite direction and forces the developer to explicitly opt in to traits. That's a bit more annoying, but it makes for much more scalable software. I don't know how many times I accidentally didn't implement an interface fully and my types didn't come through until I ended up with forcing the compiler to help out:
var _ someInterface = concreteType{}. I ended up doing that pretty much everywhere because I ran into so many times I accidentally didn't fully implement a type.I still think Go is a fine language, I just don't think it scales well to larger applications. And that's what I want to build, so I don't use Go much anymore.