federation not completely syncing
federation not completely syncing
Hi guys kinda found a weird bug on some communities limited amount of content being synced.
Example voyager (only two posts synced over the course of a day)
https://bookwormstory.social/c/[email protected]
https://lemmy.world/c/voyagerapp
Example floating is fun (only new posts are syncing no prior content is syncing)
You're viewing a single thread.
I have to update the instance to 0.18.2 sometime soon, I'll look into it then. Gut feeling says rate limits
By rate limits do you mean other instances limiting the amount of content that can be synced (ex lemmy.world) or bookworm unable to sync due to high load (not sure how much bandwidth, storage, and cou resources federation takes).
Not other instances but this one. More specifically I suspect the large instances consume all of the available federation rate limit leaving nothing left for smaller instances
Gotcha thanks for the clarification.
Ok so for starters I added a whitelist for lemmy.zip because the entire .zip tld is blocked in my network. Lemmy is now updated, I'll check back alter so see if any error messages pop up.
Kinda a follow up your theory about a rate limit seems to be spot on was checking my subscriptions and noticed the upvote counts were stuck at 0 or -1 while cross comparing to the larger remote instance they are (50-100).
After the update however vote counts seem to be working for newer posts recieved after the update
Probably fix is due to new changes in 0.18.2 which dynamically increase and decrease workers on demand
https://github.com/LemmyNet/lemmy/issues/3121
https://github.com/LemmyNet/activitypub-federation-rust/pull/42
Quick question regarding the rate limit.
Is the rate limit fine tuned to initial history sync from new federated communities (ie if you join a new community only 10-20 days of posts history will sync after which point only new posts will received and no further history will be synced).
Or is the rate limit more like the instance overall can perform a set number of federation actions in a given period, and the volume of new federated content from larger instances prevents new history from being synced.
I noticed when subscribing to new communities new posts, comments, upvotes sync pretty much instantly history seams to be the part the gets bogged down.
The second option is the more accurate description of what's going on, 0.18.3 should alleviate that somewhat as inactive instances will start being phased out now. Previously even dead instances took up time during the update cycle having all activity pushed to them even if they were dead.