There has to be a much more methodical and deliberate way of handling this, top-down. It's possible to do this when there's an actual established community (parish) for a trad order and not just a travel-to Mass, but I think St. Mary's is a community and not just a location.
Bulletins and/or other published material in the vestibule have to state that regular attendees are assumed to be part of the community and thus will be approached by others when standards are not met. Those standards have to be detailed (which I'm sure they are at St. Mary's), but it has to be announced both in writing and from the pulpit that the standards will be enforced. The enforcement will be that if compliance does not result from one warning, the person or persons will be asked to stay away until such time as they can meet the standards.
One of the first things that has to be done is registering contact information with the trad order celebrating the Mass. This is especially important for families, because parents also need to be held accountable for behavior and dress of teens. When I recently registered for an SSPX women's retreat (I ended up being too squeezed in my schedule to be able to go), it was done in a very orderly way. First I had to call them and give current parish and other relevant information to the administrator of the retreat. This made me feel very good, actually, because the retreat hosts have every right to that information, and I was comforted that they were so thorough.
Next, I received in the mail two pages of details about expected behavior and dress on the retreat -- exactly what is required of women and what would not be accepted. (There might have even been something in there about being asked to leave during the retreat if those standards were violated; I don't have the letter handy.) If I were a person inclined to ignore or rebel against regulations, the mailer in itself would deter me from going, because it was clear that I would have stood out and drawn negative and disciplinary attention to myself. The message was: We mean business, and you've been warned.
Second, group pressure -- in a parish situation -- can be a powerful thing. Again, community can make a difference if the community as a whole supports female modesty. Thus, when my sweater accidentally slips a little bit off my shoulder (and I am wearing cap sleeves underneath), one of the women at the reception, or even during Mass if I'm sitting in a pew, will gently fix my sweater. If instead it's my seminarian friend, he will respectfully say, "Careful of your shirt." I think this is all really sweet. It shows a lot of true charity and respect. The point is, that it shows that the majority of the community supports the same modesty goals as the priests do.
More on group pressure: Someone at our church recently, with permission, set out flyers in the vestibule about the necessity of women modeling themselves after Our Lady's modesty. There were specifics in there about how pants are men's dress. Some pant-wearing women objected. (Surprise, surprise) Perhaps it's just a coincidence that two days ago, rather than the usual about 8 women wearing pants, not a single woman attending wore pants to Mass. And some of the chronic pant-attired women did not show up at the usual Masses they attend. And I don't care, because Tradition is not just about the Mass.