Check out the Lynx Launcher. It's tracker free from top to bottom and gives you 5% intervals for text size per screen. I love using it and it's always one of my first installs on a phone!
Edit: You can alter icon's and icon sizes too, plus with Arcticons you've got a solid foundation for a clean phone layout.
Im on a Pixel 5 and i was also not statisfied with the different settings in the display options, but you can actually change the dpi settings in the developer options. If enabled, there should be an entry called "smallest width" which you can set to (I believe) any number you want. Try playing around there a little bit. For me at the P5, 415 is just perfect.
If you want to reset the settings, but cant remember the initial number, just go to display settings and use the sliders as in your screenshot. The DPI will reset to the default settings.