These two are similar in appearance. The rufous neck and back are typically more orange/brown. The Allen's back will be greener. but it can be a challenge to see as they both are brown and green! Their tail feathers are also somewhat different but that can be even more difficult to spot!
Rufous hummingbirds are very aggressive compared to other hummingbirds. They have a very long migration from Canada/Alaska to Mexico, and have to come into another birds territory to eat/drink while they migrate - so they can be bullies! With Allen's hummingbirds - some are migratory and some are not!
Just to make it more confusing, these two sometimes breed together and there are Allen's x Rufous hybrids as well!
During this time of year, if you live in the Rockies, it may have been a Rufous. If you live on the west coast, it was more likely an Allen's.
The best way I can get a good look at a hummingbird is to have a feeder with a perch, and additional perches around the feeder.
Ha, in that case, it is either a lost Rufous or an Allen with a chip on his shoulder. Or perhaps a lost and pissed off Rufallen ;) Thank you for the explanation!