It's healthy for a woman to be attracted (not the same as lust) to a shirtless man whose physique is the result of being physically active, just as it's healthy for a man to be attracted (again, not to be confused with lust) to a scantly dressed woman who is in good shape. To be repulsed by a good physique whose form is visible, for whatever reason, by the opposite sex is actually unhealthy.
You can disagree with me, but you will still be wrong.
With above being said, scantly dressed women are immodest and an occasion of sin, while the shirtless men are potentially being immodest, depending on the circuмstances. Women are homemakers, therefore, they have absolutely no reason to be immodestly dressed relative to the circuмstance in public. They are supposed to be dressed modestly in private, too. Conversely, men are the laborers, builders, hunters and defenders, so less clothing, in those situations, can sometimes lend to efficiency and relief from heat and humidity.