Because when you follow this dynamic? The implication is:
"Human women are so hot that even Non-Humans desire them sexually."
When it's the other way around? The implication is:
"Human Women are so shitty that their own human men abandon and replace them with anthro girls, sex bots, aliens, etc."
It always goes back to how the human twats feel. And for that reason men who say they find a non-human female attractive must be shamed. And if they're not shamed? They're outright vilified (see James Cameron's Avatar). The human species can be summed up as a vagina. It only cares abotu vagina because vagina. If there are aliens out there the best thing they can do for this species is exterminate it for it's hypocrisy and preferential pussy treatment proving its inferiority.