Lemmy is all I use, and since it's so small I just browse /all and when I keep seeing subs I'm not interested in (or a language I don't understand) I just block them so I don't see them anymore.
Then I've instance blocked a couple places because of how their mods/admins act so I don't accidentally stumble into a sub from that instance.
You could try to curate a list of subscriptions, but I just don't think Lemmy is big enough for that yet, and you'd miss new subs.
I use the web client, but it should be wherever your block list is. Webclient you have to manually type it in, there should be three sections:
Accounts
Communities
Instances
So if you can't figure it out in your app, you can just sign on via a webpage to block instances and that should effect the app since it's about your account and not the client.
I don't see a way to do it in Voyager's settings, but if you go to lemmy.world in a web browser and go to your settings, there is a "Block" tab at the top next to settings. Go to that and you can block users, instances, or communities.