Just need a well lit room for light mode and at least the white of the background bleeds into black text making it grey rather than the blurry mess of white on black.
I have astigmatism too and for me it looks just as bad when inverted, a blurry mess of black on white.
To me, what makes one a better choice vs the other has more to do with my environmental lighting.
If I'm on a place with a lot of light, the screen being black makes it have reflections so I prefer black on white in those cases.
If I'm in a dark place, the bright screen makes the screen itself the primary source of light which is not good for my eyesight, it can make my eyes feel like burning/tired after a while.