There's some cultural subjectivity to it obviously. Some cultures would find him to be effeminate. And there's no doubt he'd look even MORE "manly" if he had facial hair ... unless of course your culture consider that to be a sign of being an uncivilized barbarian.
.
For well over 100 years, shaving has been the hallmark of civilized man, so that when people wanted to attack civilization, the first thing they did was to grow beards. You rarely see men wearing any beard at all in the early 20th century:

Look at this orderly, cultured gentleman from the 1950s.

This was a time of peace, order, morality and progress of every kind. Almost all men looked like the ones above. The clean, shaved appearance represented the cleanliness of thought and orderliness of life.
To bring down this civilized order, one of the first things hippies did was to stop shaving.
These people are barbarians in the truest sense of the word, meaning not just that they are uncivilized themselves, but that they are in fact waging war against civilization and locked in a deadly struggle against it. Unfortunately people like this won the war, and what we are living in now is the ruins they have left of what the men above created:

This phenomenon exists to this very day, although the beards have become slightly less barbaric than the guy above. You see it in the constant need of Portland hipsters (communists) to wear at least some facial hair, as if it were some badge or symbol of their war on civilization:

I realize this probably sounds like an overly simplistic analysis, but I believe it is a matter of simple observation that in modern times a shaved face has been a symbol of those who build up, while the beard has been the symbol of those who destroy. Why this is the case, I can't say for sure, but I think part of it has to do with the fact that shaving is itself a product of civilization. To shave, you need finely-sharpened razors, soap, shaving cream, or electric devices -- all of which can only be produced by civilization. Any savage running around naked in the woods can grow a beard; in fact, growing a beard indicates either that you do not have access to such products of civilization at all, or that you eschew the use of such technology and prefer to look like a savage.
A man is immediately identifiable by his beard, or lack thereof, whether he lives in a civilized society, or takes advantage of what it offers. And someone who lives in a civilized society and despises the technology that such a society creates shows that he despises the advancement of that society, and therefore civilization in general.