Fans suffer when their team loses. People suffer when they lose a family member or are fired from their job. It's not a competition as to who suffers more. Have some empathy.
I do the job I actually like, and it hurts when I have to do some unrelated shit instead for a while, because my employer says so. Why would professional athletes be different? Money doesn't justify everything.