I once asked a question, everyone thought I made a bad joke, down voted, I edited it to include that it was genuine curiosity, got down voted even harder and called stupid, my noddle still doesn't compute the situation.
Sometimes you start with the person you replied to down voting you because they think you disagreed with them, then others read your comment and up vote it
Nope. That's rare. It happens here more often than on reddit but it's still rare. If you're positive karma, most folk who don't agree with you will just pass by or answer. If you are negative, straight to downvote it is.