This is a known issue that has been fixed in Lemmy version 0.18. However some instances are still not updated to that newest version. Once your instance owner updates to version 0.18 this should no longer happen to you. Source
Although some instances may also be refraining from the 0.18 update. Most notably Lemmy.world, which is keeping away due to the removal of captcha, and plans to update when 0.18.1 comes out with a restoration, or new anti-bot system.
I also constated that sometimes, the page reloads/scroll on its own while I am watching, it's very confusing.
But I can't find a pattern to when that happens...