I agree that we are too accustomed to comfort in this present day and time. Many of us, myself included, find it almost unbearable to wear anything that offers any tension around the waist or elsewhere. Who wouldn't feel more comfortable in p.j.'s or loose fitting pants and baggy tee shirts? We all do.
It's been a pet theory of mine that all of life is about 'resistance' one way or another. We resist all that causes us to feel a tension within us, resisting temptation creates a feeling of tension, and even the wearing of more modest, appropriate clothing creates a feeling of discomfort, ie., tension.
I can also agree with the concept that in this day and age, dressing out of the norm not only causes attention to be paid to oneself, but might also create a social barrier between people who might otherwise feel more comfortable with one another.
On the other hand. many secular types are fascinated by the Amish and others who live outside the mainstream in a modest and plain fashion. If you dress like everyone else, you also may miss opportunities for evangelization because no one can tell that you are a serious Catholic.
God Bless,
MrsZ