Planning for the future - a flaw in the design of Lemmy (and Kbin)
Planning for the future - a flaw in the design of Lemmy (and Kbin)
Hey all, so I've been trying to embrace the fediverse life. My background - I've been on the internet since pre-WWW, so I've seen it all.
I think there's a structural issue in the design of Lemmy, that's still correctable now but won't be if it gets much bigger. In short, I think we're federating the wrong data.
For those of you who used USENET back in the early days, when your ISP maintained a local copy of it, I think you'll pick up where I'm going with this fairly quickly. But I know there aren't a ton of us graybeards so I'll try to explain in detail.
As it's currently implemented, the Fediverse allows for multiple identically named communities to exist. I believe this is a mistake. The fediverse should have one uniquely named community instance, and part of the atomic data exchanged through the federation should include the instance that "owns" the community and a list of moderators. Each member server of the Fediverse should maintain an identical list of communities, based on server federation. Just like USENET of yore.
This could also be the gateway into instance transference. If the instances are more in-sync, it will be easier to transfer either a user account or a community.
This would eliminate the largest pain point/learning curve that Lemmy has vs Reddit.
Open to thought. And I'll admit this isn't fully fleshed out, it was just something I was thinking about as I was driving home from work tonight
Lemmy is good, but it could be great.
I see this suggestion as problematic and recreating a problem reddit had. One group could lay claim to territory and everyone was stuck with however good or bad the culture was in the sub and however good or bad the mods were. There were some places with mods on a powertrip creating an exclusionary or outright toxic environment.
One group could lay claim to territory and everyone was stuck with however good or bad the culture was in the sub
This problem is easily surmountable with a new community name. Its not like it doesn't happen anyway because for example c/trees isn't about trees across multiple instances. Also i think 'syncing' a community could be optional decision between federated instances. If one instance grows to have values that disagree with your instance community you no longer sync that community and possibly defederate.
There are better solutions to the problem. For example, letting communities follow other communities. It's simple and flexible.
Take a look at the discussion here https://github.com/LemmyNet/lemmy/issues/3071#issuecomment-1595303910 or my diagram of one proposed solution.
This is genius, please devs 🙏
What if a post is acceptable by the rules/mods at instance b but the post breaks the rules at instance a?
Options:
- Instantly stop syncing? Seems dramatic
- Mods align rules? Utopia
- Mods keep rules misaligned and users seek out preferred instance? Most likely outcome but this would still create a weird multi-tonal vibe in 'mixed' communities at the user level experience unless instances can find a way to express themselves with a localised 'style'.
Maybe we could add some metadata, like tags, to allow them to cooperate and just block/allow certain tags, that the other sets.
Unless I've misunderstood something, community names are already globally unique and multiple identically named communities cannot exist, you're just not looking at the full name.
[email protected] and [email protected] are different names, for example.
Sometimes we ignore the part of the name after the @ if it's unambiguous, but it's still there.
tl;dr: I agree with OP that the ability to have multiple, identically named, disconnected communities on different instances will be a severe detriment to the adoption of the Fediverse by the general public.
community names are already globally unique and multiple identically named communities cannot exist, you’re just not looking at the full name.
That's part of the problem. The simplest UI generally exposes the community name without the instance, !memes for example, but the backend is really [email protected] which is an entirely different community from [email protected]. Now, that's not really a problem - memes are memes. But what about a community for Edinburgh, UK? There are already two - [email protected] and [email protected]. That's going to be an issue because if you choose one to participate in, you'll miss all of the content in the other. If you're a member of, say sopuli.xyz, you won't even know that either exist because their community search doesn't actually search all instances and might start a third. The whole idea of the Fediverse is to have a federation of instances which share information, and there is already talk of the biggest instances potentially creating a problem with the democratic ideals of the system (6 days into the reddit migration and three of the largest instances have defederated from one another). To have a thousand instances each with their own !Photography or !ManchesterUnited community dilutes the content and interaction.
I agree with OP ( I actually don't know how to link to a profile yet or I'd tag @TerryMatthews) that there should be some cross-linked mechanism to merge identically named communities across instances. There could still be detached instances - defederated content would not have their content propagated - but the content for each unique community would be co-mingled.
I would expect that moderators would be limited in scope to their own communities. So a mod from feddit.uk could block a non-instance post or user on their instance but it would be present on other instances. They could also block a local post on their instance and it would not be propagated at all. Pinned posts get a little more hairy - would every mod have a separate set of local instance pins? I would think that would need to be the case. The issue of sidebars is also an issue.
The "two Edinburghs" situation already existed on Reddit. You'd get slightly different "competing" subreddits. They'd have to differentiate their names a bit but it still happened.
To flip it round - there was an issue in Reddit that whoever first set up a subreddit with a given name then owned it forever. Let's say I got there first for /r/london but then I'm a twat and either create a community of horrible people or fail to build a community at all. Everyone in London who wants a city subreddit is worse off, and at best someone has to come along and make a different subreddit with a different name to fulfill the same person. Not having this single namespace with "first mover" advantage is good and democratic. And all we have to do is pay attention to the bit after the @ sign.
This kind of reminds me of Android vs iOS. With great freedom also comes fragmentation.
As it’s currently implemented, the Fediverse allows for multiple identically named communities to exist. I believe this is a mistake. The fediverse should have one uniquely named community instance, and part of the atomic data exchanged through the federation should include the instance that “owns” the community and a list of moderators. Each member server of the Fediverse should maintain an identical list of communities, based on server federation. Just like USENET of yore.
Hard disagree. This allows abuse by moderator abuse similar to how reddit does it. Ideally the UI would allow you to create collections of communities based on your own groupings. And search could be expanded to find more similar communities (ie: based on keywords).
The way loading/finding and joining a totally new community into a new external instance is kinda buggy.
Yeah, basically we need somthing like public and private multireddits. Public, so you can sub to one thing to get content from many similar communities, and private, for people to organize their own stuff.
Yeah agree totally. Kinda what I was trying to get at below. https://lemmy.pub/comment/9124
If you disagree, how would this be different from google then?
Sorry. Was getting late and comment submissions have been weird.
Personally I am hoping for 2 or 3 changes thst I went into here.
https://lemmy.pub/comment/2889
-1. Create multi-communities thst you can group together multiple different communities across instance into a single place based on whatever criteria. Basically like multi-reddits and allow them to be public/shared or private. This would allow the load to be spread across instances and different moderation strategies to take place without a single community becoming some weird power grab.
-2. Makes links to other instances and communities always open in your instance so you dont have to constantly create a https://my-instance.com/c/[email protected] style link. Basically what this site is doing, but integrated into your own communities section.
-3. Like above, Improve search and communities to index basic stats before you have to externally find and, search multiple times to get your instance to discover the communities then open the federated link an subscribe. And when on a multi-community aid it discovery of new additions with a basic keyword search. This one is probably the most difficult and may scare some folks from privacy standpoint. But could be mitigated a bit if you can make option one public for others to subscribe to or use mutlis they didn’t compile themselves.
I'm one of those USENET greybeards and I think this would probably be a mistake. If you let a name be uniquely claimed by an instance, how do you decide which instance gets to be "in charge" of that?
Better IMO would be to update the various interfaces to be much more explicit about including the instance name along with the user/community name. So that it's always clear that a user or community is at a particular instance.
We do still need better migration tools for moving users and communities around, though.
Reading this just gave me an interesting idea, when you start to post a link that's already been posted in another linked instance, it will start to show you that it's been posted elsewhere to different communities in other instances (on Lemmy, I don't know about Kbin). This clearly shows there's functionality there to look around when posting links, so I wonder if similar could be implemented when creating communities.
If the interface told you ahead of time that the community you were about to create has already been created in other instances, you wouldn't be prevented from going ahead & creating your own version, but you'd be more readily aware. Honestly a win-win approach imo, considering it would help you find a community you may have been looking for but didn't think existed, and it doesn't keep you from trying to make your own anyway.
couldn't have put it better myself (also a USENET oldie but w/o the beard)
It’s not about allowing a single instance to own the name. The name would belong to the federation in a global namespace.
A possible scenario is to define multiple namespaces. Each namespace can be local to a single instance, or shared between many. Within each namespace, a single community name is unique.
In this model, each instance would have a namespace that it owns, and the ability to participate in many others.
The trick is in how we name the namespaces and communities. We could do this the USENET way and do something like <namespace>.<community>, so beehaw.gaming vs. global.gaming. There are other models that could work too.
I'm not sure how that would be different from what we've already got.
IMO the main feature kbin/Lemmy are missing is an equivalent to "multireddits." That would allow multiple communities to be seamlessly aggregated for a user, they'd see all the content blended together as if they were one. I remember seeing a Codeberg issue over on the kbin repo discussing how to implement that, and I'm sure Lemmy's devs are working on it too, so that feature will probably come along fairly soon. Then it shouldn't matter much if the same subject has had multiple instances set up communities.
I think that would defeat a piece of the point of a decentralized system. In the current design, what will naturally happen is that if one instance has all the good content on a particular topic, most users will gravitate toward it anyway. We can read across federated instances anyway so I, a kbin user, have no problem reading something on lemmy like this.
Then let's say one day [email protected] gets taken over by people who want to post stuff I don't want to see. If I miss how it used to be here, I could go make [email protected] and it would be fine.
I'm not understanding how this would work with instances who wish to defederate and segregate their community? It seems like an "all or nothing" approach that instances who have defederated already wouldn't be on board with... For instance what happens if beehaw owns the "gaming" community, and then defederates from lemmyworld. Lemmyworld users just no longer have a "gaming" community?
Best example and reply IMO
I figured it was relevant :)
It seems like this concept is “orthogonal” to the current federation concept, proposed by the ActivityPub protocol.
In the proposed case, the instances act as a pure “computational and storage fabric” or some kind of a “cdn” of a service, without any “personality” incorporated in them.
So I would not say that this is “better” or “worse”. It is just another concept.
Yep, I think this concept is actually going to be necessary moving forward, some kind of caching/relay infrastructure, owned by others but lightening the load for other instances and providing a good starting point for newly created instances that just want a stream of content right away.
They may have the same name, but they have suffixes that don't actually appear, such as @lemmy.world @lemmy.lm. I oppose this idea. Because when the servers do the defedere people don't have access to that community. We have recently seen an example of this. It's okay to have more than one community with the same name, but a lemmy grouping feature can bring. In this way, we can eliminate clutter by grouping communities with the same name. At the same time, the fact that different communities with the same name have different moderation understandings provides people with an alternative in community selection. Reduces moderation pressure
The way I see it everyone naturally assumes we're trying to recreate Reddit but with distributed computing.
I think instead we should be trying trying to create something that gives us the community and communication that Reddit gave us, but democratically and without reliance on or control from any one organisation.
This is going to result in some things that work differently from Reddit. We should work to make the experience smooth and intuitive, but it can end up with a different way of working.
100% agreed. I'm not advocating we "clone Reddit", however I do think we should think about and take meaningful steps to improve accessibility to non-"techy" people even if that means borrowing a few things from Reddit here and there.
Because let's face it, Reddit wasn't a whole-cloth original creation of spez and kn0thing. It's bones can be traced back to Digg, vBulletin, earlier BBS incarnations, in some respects even USENET - especially the way users can create topics/communities/subreddits on their own (yes, I know this isn't how USENET works now, but I promise it used to work this way if you were outside the main controlled newsgroups).
I'm a smart guy. I've got a lot of years of internet experience. I can make Lemmy work, and find content on it. It's cumbersome. My wife, is very techy by any reasonable standard but not as much as I am, has difficulty using it. She finds the structure unintuitive and confusing.
If those of us participating in this thread are the 0.1%, she's the 1%. To me, this moment, this movement, is about ensuring there's a place where people are free to discuss things that monied interests can no longer control. That's what makes the fediverse great - we can spread the load and demand out and make it manageable for normal people to do this.
I don't want another schmuck coming along telling me what ad I have to look at, or what I'm not allowed to discuss, or what app I have to use ever again.
I'm not the smartest guy in the room, I'm not claiming to have the answer only a suggestion. However, I am confident that this is a problem we need to tackle in some way if we ever want to achieve growth in "normal users".
Completely agree that there needs to be some strides in usability. I'm in exactly the same situation with myself and my wife in terms of what's needed to be able to recommend Lemmy to her.
I just wanted to get people thinking about a "product" direction and set of solutions to these problems that weren't only aimed at replicating Reddit.
As I've said elsewhere the communities sharing the same name is just a temporary problem. As the system grows one of the identically named communities will become dominant. Because of that small new communities that want to have the same topic will be incentivised to have unique name just to be able to be found.
Good point. Also on reddit there are/were multiple subs for one topic with one or two gaining major traction over time (while others continue to exist alongside)
Ether that or allow formeta-communities so that the 500 duplicate copies of "Technology" can self aggregate.
Which I don't really think we want because it's all about context based on the instance. [email protected] would be all about solar tech whereas [email protected] would be more science and consumer electronics.
I think the mods of each sub should agree to aggregate using an "invite". It could also allow subs that have slightly different names on different instances to team up, if they are essentially the same thing.
I think that would self-correct when such a thing would be implemented. They'd just get more specific names, like solarpunktech or such. It would be an issue for existing communities like that though.
I disagree respectfully, as I think this is a feature and not a bug of a federated structure. It's well known that reddit suffers from the "20K Law", which is that "The quality of any subreddit drops off a cliff after it gets more than 20K subscribers". Which is likely because that is the limit of effective manual moderation.
So, having multiple communities on the same topic would be a fundamental fix to that issue, as instead of one giant community, instead you get different, smaller communities with different culture on the same topic, whose users can still talk to each other.
I think the current system is fine as is, we're not trying to remake a better reddit, we are trying to be better than the limits of reddit.
I think it's a feature, not a problem. If you have multiple communities for popular topics, if one or two of them turn toxic or have unpopular moderation policies you have other places to go. Think of it like forums, popular topics had many forums to choose from - but each had slightly different cultures. Also since forums could individually be quite small, they were often a lot friendlier.
Federation makes it easy to explore different communities, but we don't need to import the bad parts of centralisation.
I agree that this is a useful property of Lemmy once things get going but I think right now what we need if Lemmy is supposed to be a going concern is a less fractured landscape so people can actually find the content they are looking for. Some kind of global directory might be really useful there, just to make sure people interested in similar things can find each other.
The easiest to search directory I've found is https://browse.feddit.de/ (although I don't think that picks up Kbin). A directory doesn't need to be centralised, it just needs someone to make a good one. I think what will naturally happen is one or two communities for a topic will become prominent and will emerge as the obvious place to go, and I think it's best to let that happen organically.
Having sibling communities would be another approach.
Each one is still homed on their instance, but if the instances are federated the posts from the other instances automatically flow into the community. They would still show the originating instance, but content would be comingled.
Along those lines, what if communities could suggest other related communities, and clients could (by default, but optionally) also show posts that the communities your subscribe to recommend?
It's a little like soft federation, but on the client side - or, think of it like an automatic way to make multi subs while still allowing users to change it, and giving mods a less nuclear option if they don't like how a community on another instance is going.
Wow yes this sounds great, if a community or commune becomes big enough, multiple instances can share the load and carry content for said community, provided that the people running the instance and the people modding the community can come to an agreement, and for private communities, well we'll see further opportunity for access and more vetting procedures etc
The biggest problem with this approach is the potential headache for mods, and it might actually make things more confusing for the general public.
But OP's approach has a technical issue with a federated system: when two instances federate and each has a community of the same name, who gets to keep it? There's no central authority for such things--that's the point of decentralization.
So I guess it's just a complicated situation.
This could also be the gateway into instance transference. If the instances are more in-sync, it will be easier to transfer either a user account or a community.
I think this is perhaps the most compelling part of your thinking here, but at the same time, I'm more drawn by the idea of improved portability of instance information than the synchronization.
I may be mistaken, but I think a fundamental aspect of the federation model is the independent operations of each community while enabling their intercommunication, and the sort of synchronization you suggest would be a violation of each instance's autonomy to a degree. Not to say it's a bad idea, just that I think it would better fit a different networking model than what I think the federated model seems to be pursuing.
USENET was a mess too, though... The entire Alt.* hierarchy was an attempt to route around censorship and I'm pretty sure I recall lots of duplication of groups. /derail
Man, as an outsider, I thought all newsgroups began with alt.*
I understand your idea, but I think it would defeat the purpose of the fediverse. It would create single points of failure that are un-correctable.
I also think many people forget that Reddit never functioned any differently. Everyone seems to have forgotten (and I’m not saying you have!) that there are and were always multiple subreddits for any given topic. With slightly differing names. The only reason people are forgetting this is because eventually one or a handful became pre-eminent and the others died or became transformed into something more niche.
I think it’s a problem that will ultimately correct itself, but I think a tags based system, like hashtags in Mastodon, would be a better solution for tying communities/magazines together through metadata.
To your point, Seattle had 2 subreddits due to disagreement on moderation lol.
Another weird example would be r/soccer and r/football where soccer ironically became the defacto.
Communities should have categories/hashtags that users can optionally sub to, like the 'metacommunities' like plz1 said but optional and multiple. Mastodon does hashtagging and can be done on a post by post basis. The forum software Flarum has a 'tag'/category system and an additional hashtag system, so what I'm thinking of is more like the Flarum system since it would be awkward to hashtag every single post in a community/magazine/whatever.
So if I wanted to just get solarpunk tech I'd sub to that, but if I wanted that and even moar I'd sub to a generalized Tech tag. Make sense?
@Stardust this is a great approach I see suggested often. A 2-dimensional identity for a community; one through tags, another through names/policies/wikis/sidebar info. Like, you might have users see all the time and identify with from a specific sub, and also have labeled/mechanical community interaction with much less familiar people in the tags if that makes any sense. Kind of like how Reddit would recommend related posts from adjacent subs I suppose, but this would be on a user-generated level.
I am glad I read through the comments first because this is what I was going to suggest. Because you may not want to see all the comms post because some are more friendly then others and some may only post in an language you don't understand. With tags you can discover all the comms or even create one feed with comms like #tech #English #Japan you never even sub to them unless you need to manage those particular comms/posts with those tags.
We could make it possible to group tags so there are multiple feeds and you can pick to see all feeds or one or many that you have created. It will be like groupings all of the comms/post you like together to one community but making it very personal because you may only care about tech in Japan but you want to have all the news from your country and maybe do not care what country memes comes from. But if you really want to mange them by comms then that should be an option to. So the tags will be a tool to discovering comms instead then.
What would prevent tag pollution? Eg tagging non tech items with #tech, either maliciously or ignorantly, or accidentally. Or ambiguous crossover - tagging cute cats #tech because it's sitting on a computer...
Is tag moderation a thing? Could community up/down vote individual tags? +5 cute +8 aww -3 tech? You could filter content based on tag rating. Not sure people would go to the effort...
This is the best solution I've heard so far. Any server could have their own Technology group. Using Federation, anyone from anywhere could subscribe to each of them. Or, instead of subbing to each of them you just sub to the !tech tag, and you automatically get content from all of them. When you start a community you apply any tag you want to be included in.
To me, the instance should be mostly invisible/seamless. Subbing to tags instead of instance communities puts the focus on the content rather than where the content came from. Tags would make one large meta community that simulates how that other site feels, but with the option to still subscribe to a specific community if you ended up liking it more.
Say for instance one of the !tech groups ends up with really good content and discussions and the other smaller ones end up with a lot of duplicates and low quality comments. You'd easily be able to see which one you'd want to sub to directly. In this way tags would make community discovery much easier. Instead of having to seek out 10 different groups on 10 different instances, you sub to a general interest tag and either that works well enough or you discover the one you like the most and sub to that one directly.
@polygon I'll be interested to see this happen in the threadiverse side of things (all these link aggregation protocols like L/k right now). In the larger fediverse, this (tracking hashtags) is basically the number one way to do discoverability (i won't get into why but suffice to say straight search isn't fully supported technically and normatively). All the microblogging protocols (masto is one) allows you to follow hashtags (and the contents will show up on your timeline without having to follow accounts), though how it's done is different based on protocol. I'm curious to see why L/k doesn't automatically allow user accounts to do this, perhaps that was the whole point of the comms/mags.
Yeah I think tags or some other way to congregate similar content from different instances would improve my experience a lot. I got like 3 same subs now on 3 instances and all their content is small, if those three were linked somehow say through a tag, they could interact with each other a lot easier and it would seem like one bigger community.
I'm personally ok with multiple technology instances. It was weird at first, but I think that is by design.
I do agree that user management needs a redesign. I have three fediverse usernames that I created because I was a noob and everytime it asked me to login I would get denied and create an account. This would be absolutely awful for the normies.
This is where IPFS might actually be useful. Also this might help fight the trolls.
This could also be the gateway into instance transference. If the instances are more in-sync, it will be easier to transfer either a user account or a community.Indeed, does it not make sense in a fediverse where you can forward or change your account to another instance a community, be it called magazine or lemmy can change instance as well.
It would also be a protection.
Already now we are seeing some instances of lemmy's / magazines growing larger than others e.g. selfhosted on lemmy.world vs lemmy.mlImage in a year time if the largest of the communities would suddenly drop out (database corruption, server takedown, admin issue) again all the knowledge / posts are again lost and difficult to recover.
Or does it already work that way and I'm still not really grasping this hole fediverse thing?
My understanding is that if an instance suddenly dies, all the federated instances that subscribe to its communities will still have the text content because they store copies locally. So knowledge should not just go away. Media is a different story though.
I think new posts/comments in those communities would then not federate at all anymore since the host instance would not acknowledge them. So the communities turn into isolated local ones.
If the host instance comes back and the communities are re-created, they’ll be empty on the host instance but I think other instances won’t delete the old content unless explicitly requested.
How is this scalable and sustainable? If every instance caches every other instance then it'll blow disk usage out of the water.
I understand what you’re saying but i have a different idea.
First, user accounts should probably be federated. Or federated accounts should be easily identifiable versus non-federated. Federation is pretty easy if you tie it to google, Facebook, Microsoft saml/openid. I’m fine with those options but I understand how others may not.
2nd, I think magazines should be collapsed until they are not.
For instance, pics.kbin, pics.lemmyworld, and pics.reddit2 should show up as the “pics” magazine. If kbin decides to defed, their content now appears to everyone as pics.kbin.
This adds a layer of abstraction that only appears when it’s relevant. Users could of course decide to display this info if they wanted, but by default it wouldn’t show up.
Moderation is more difficult, but I think federation has a place here too. A magazine could decide to federate, and the mods, with federated identities, would then be able to do the needful across instances. If things don’t work out they could defed their magazine.
I made a similar comment as OP did in r/Redditalternatives, but I like your idea even more. It's the best of both worlds if implemented correctly. Can it be done even without making things too complicated?
It's interesting how this issue is new (in my experience) to the Fediverse with Lemmy. With Mastodon you essentially have this uniqueness functionality built in. Because a person isn't likely to post content to multiple accounts under the same persona. Instead someone will choose their instance and username and that's where you can find them. Sure a user might have multiple accounts but I think usually it's to separate multiple online personas.
I think there is use in having same named communities in Lemmy on different servers because they could be for different purposes.
But I can see multiple duplicate communities with the same purpose being on different servers causing fragmentation. Maybe we'll end up with popularity wars and eventually one duplicate community will give way to the other?
I'm already seeing the duplicate community issue. I am a weekly reader of One Piece and currently, there are 4 communities for it on Lemmy alone. Its not helping the community come together.
Is this similar to the global DNS network? There would need to be a protocol to exchange and keep the list up to date
I can't form an opinion right now.
If all similar communities appeared combined into a single community I'd still be likely to want to still filter out sources.
But at the same time sometimes a place to see them all together sounds appealing.
It seems that I want both, but probably what we have now is the most flexible, forcing less limitations.How does that not defeat the entire purpose? You're suggesting one global 'sublemmy', as it were, with one global team of moderators. How is that in any way different or better than what we just left behind?
no it doesn't [email protected] is a different name than [email protected]
You also need to remember that these platforms are based on the W3C standard for social media, ActivityPub. Lemmy & kbin don't want to go too far out on their own such that they no longer function with federated services as a whole (mastodon, frienidica, Peertube, etc.)
not sure that solution is a good one for this environment. I'm new but from what i've seen the concept of moderation is different and your solution is trying to engineer a reddit-like moderation design to an architecture that is fundamentally not reddit-like. Moderation here is at the instance level, not the community level.
This would be excellent if done right. What I am curious about is, where this will be implemented? On the protocol level in activity pub or with each GUI (mastodon, pixelfed, lemmy, kbin etc) need to individually implemented it?
@[email protected] I think fediverse problems could be solved by apps. An app could try to search the community you're interested in across all instances and merge the results.
Apps could also solve problems when you have multiple accounts.
It could merge all feeds from all accounts and reply from the right account.Nah, think of it like subreddits, there are many subreddits for technology, people will use the ones they like and you can easily subscribe to several, it's fine.