Taylor swift was born in a family of stock brokers, fully supported by her rich parents even at the age of 10. Kanye on the other hand started in the ghetto with nothing, not even talent.
But at the end it boils down to the fact that me and most men I know do not ever listen to Taylor Swift, they could however stand Kanye West at a bar or a similar social situation. What he did was wrong, but at the same time it was entertaining.