You're not lying. Looking for the hidden quests without knowing if you had already done them and just sort of blindly farming for a bit took by far the most time.
The last one I was missing in the end was the crumpled note in Caldeum Prison which has a low drop rate to boot.
I think my last one was glittering ore in Hawezar. I used a guide for easy to miss quests and I also had a good portion of them tracked through the D4planner map, but it didn't had all the quests at the start so a lot of them were added later, combined with me maybe forgetting to check some off myself, creating a bit of a convoluted mess. Still, I was able to narrow it down to some potential ones and just farmed the locations for several days, which every guide lists something else too. Absolutely painful. I knew the UI was bad but I can't believe they skipped on something as important as this.