My wife and I didn't even realize this mechanic for a while and were confused why our horses would go at different speeds all the time. We figured it out eventually, but the mechanic is unintuitive and not at all user-friendly. I looked through settings hoping to find some sort of radius setting to reduce the mouse drag distance and was disappointed to not find anything.