Well there are different rules on different servers, for a start. Which ones they're federated with can potentially make a big difference.
And then of course if you're just relying on All, you'll miss out on stuff that a bigger server might see because nobody from yours has subbed to it yet. But that's sorting itself out over time, and you can always speed up the process by making sure to regularly do a trawl for new interesting communities to pull in with a tool like lemmyverse.net/communities. Absolutely not a reason to sign up on a big server, just a reason to be more proactive if you're on a smaller one.
Wait... ugh, someone needs to make a cute little graphics animation to explain this shit.
I'm on lemmy.ml. When I choose to see "all," is that just listing communities that other users on lemmy.ml subscribed to?? I thought it listed content from all other servers connected/federated with lemmy.ml??
It's posts from all the communities lemmy.mlknows about, as opposed to all of the communities everywhere in existence. Given enough people and time the two lists end up being basically the same but obviously during a period of rapid growth like right now, new communities are popping up every second so it's gonna take a little while until all the major instances know about them.
Tools like lemmyverse.net let you keyword search communities based on your interests so you can find new things to sub to. If you're the first to do so from lemmy.ml, congrats, your server now knows about that community and everyone else on there will start seeing posts from it in All.