UPDATED Oct-3: Issue with accessing discuss.online from 3rd party clients
We are aware of an issue with some clients not being able to sync (receiving 502 errors, etc..). It does not appear to affect all clients and the web view is still working. We will update this post as we know more and when we have resolved it.
Updates:
09-Sept 11:19 PM UTC
We attempted to roll back from Lemmy v0.18.4-beta.8 to v0.18.4 (as well as even earlier beta versions) but the service failed to start. We've reverted back to v0.18.4-beta.8 for now and will continue investigating the cause of the sync issues.
10-Sept 3:30 AM UTC
And... we are back, where we started. If you hadn't noticed, we were down for almost 2.5 hours while we tried to rollback the database migrations introduced in the beta releases so we could go back to the stable v0.18.4. When that failed, we tried to restore from the backup we took just prior to the rollback attempt. When that failed, we used Digital Ocean's restore to point in time and created a new DB cluster and switched to it. So... we are back on v0.18.4-beta.8 again, with the expectation that 3rd party clients are still broken. Unfortunately, we are not sure what we can do about this as we believe it is something in Lemmy that is breaking it. The mitigation we have put in place to avoid this going forward is that we no longer use the "latest" tag when loading a docker image and now hard lock to a specific version. We will update that version in a more intentional manner going forward.
10-Sept 4:15 AM UTC
After some additional digging through the Lemmy diff for beta.5 to beta.8 (and then looking through the PRs that were merged to create this diff), the issue is indeed with the Lemmy code and is even called out in one of the PRs,. What this means is that the 3rd party clients that are not able to access the site will need to update their code to fix things on their side (or else they will be broken when v0.19 is released and all other instances update to it).
16-Sept 11:23 PM UTC
Both Sync and Mlem have now released new versions that resolve connectivity issues. If there are any remaining clients we suggest you reach out directly to them to ask them. We'll be unpinning this post in a few hours
27-Sept 10:30 PM UTC
We have made the decision to update to the RC (release candidate) for v19 as the beta we were previously on was missing some much-missed admin tools that we need to run the site with. This update appears to have affected 3rd parties again (I have tested Mlem and Memmy and get errors about authentication, however they still seem to load?). There also appears to be an issue with the home page loading that we are currently looking into how to resolve. As before, we don't anticipate any way to roll back. To steal a line from my daughter's favorite movie: Keep Moving Forward
28-Sept 12:03 AM UTC
We found an issue with the markdown parsing that was causing the homepage sidebar to throw errors. We've resolved the issue and will update the core developers of Lemmy.
03-Oct 1:28 AM UTC
We have implemented a workaround that should have fixed all apps. Please let us know if you are still having issues with an app (we may not be able to add a workaround for it, but we would like to at least track the ones not working)
The worst thing about this problem are the apps. I'm using jerboa and voyager webapp. Rather than catching and throwing an error, both of them are crashing completely instead. They really shouldn't be doing that. I've had to disconnect from the wifi and remove the discuss account offline so it wouldn't crash on launch.
alexandrite.app - login gives a message that it’s only compatible with 0.18.1 or newer and fails to login
Voyager - was already logged in. Timeline shows but list of communities won’t load.
Thanks for your efforts but as a user this seems like too much hard work. I’ve reluctantly created an account at lemmy.world. Is there no way that we can’t go back to 0.18.5?
I made a similar mistake on my instance. My database was corrupted, and after some trial and error, I performed a conscious upgrade to :latest but now I can't use Sync for Lemmy. /sadface/
OK can you explain how to make a post? Do I navigate to subscribed communities, click on respective community then post? I have only ever posted using apps, so am confused.