Oremus I agree with you that there's plenty of vanity to go around. Among both men and women. Men aren't exempt from their brand of gym variety vanity. However, wanting to look and feel one's best and fit properly into one's clothes is not automatically vanity. We do have a basic right, with the dignity of being human, body and soul, temples of the Holy Ghost, to want to look and feel our best. There's nothing wrong with that until it is given higher priority than it deserves. It can cross the line into vanity, quickly and easily, but so can all things. Our ability to abuse our goods, temporal and spiritual, doesn't change the fact that those goods are good. If a person exercises for a less than perfect reason, the exercise itself is still good for them. It's their own perceptions and intentions that need correcting, not necessarily the exercise.
I know where you are going with sports but by that reasoning a man should not play sports either. He has just as much duty towards his family, if not more as head and leader. And "playing sports" covers such a wide variety of activities... I don't think women should play football, for example, and I do think many sports are very unfeminine, but neither would I say they can't play any sports. What about tennis? Or skiing? Or swimming? There are many sports whose benefits easily outweigh their risks.
I think the biggest problem with the modern view on exercise is that it is in mixed company and public. Honestly, to me, unless you engage in a social sport, like Badminton or something, exercise is best kept as a more private activity.