I have a home server and I have some HTTP services running on it. I'm thinking if I should even bother with HTTPS, as I'm already using tail scale which should be peer-to-peer and encrypted. So I shouldn't worry about any men in the middle.
Your VPN provides the former but not the latter. That said the odds of there being an issue in this regard are so slim as to be zero, so you'll probably be fine.
It can still have issues with potential attacks that would redirect your client to a system outside of the VPN. It would prevent MitM but not complete replacement.
Do you have any devices on your local network where the firmware hasn't been updated in the last 12 month? The answer to that is surprisingly frequently yes, because "smart device" companies are laughably bad about device security. My intercom runs some ancient Linux kernel, my frigging washing machine could be connected to WiFi and the box that controls my roller shutters hasn't gotten an update sind 2018.
Not everyone has those and one could isolate those in VLANs and use other measures, but in this day and age "my local home network is 100% secure" is far from a safe assumption.
Heck, even your router might be vulnerable...
Adding HTTPS is just another layer in your defense in depth. How many layers you are willing to put up with is up to you, but it's definitely not overkill.