Good question, especially since folks on this site often feel unliked unconditionally.
As to not liking someones face, of course we wish this were not so, but many of us do it just the same. For instance, notice that men in elected office in the US rarely grow beards or mustaches, as this would, in our culture, give an impression they are hiding something.
People with relatively small eyes, according to various studies, are not considered as trustworthy as people with eyes of normal eye size proportion. This seems to be an inherent distrust we foster.
Then I have read of people who distrust red haired women. I hope this is very rare. Surely this is a taught attitude. Sad but it happens…