It's how you wear them. I have never been particularly elegant, and I think that if I had tried to claim to be more womanly in a skirt than all the other more fashionable and, yes, feminine women in the office who wore trousers, there would have been some politely stifled chuckles!
It's not about you versus them.
My point is that I would hardly have been regarded as a more feminine and womanly specimen than the other women in the office who wore trousers. See, they wore make-up too and had nice hair. I don't, and my hair is a law unto itself. A skirt doesn't make me more elegant and ladylike!
It's about pants versus skirts.
I don't see an opposition between them. Most women who wear trousers also wear skirts. Usually quite short ones though.
Interestingly, as I recall from my time when I worked in an office, there were a couple of women who
always wore trousers. They were Muslims, and they wore veils as well. Not feminists!
"In the office"
Doesn't exactly sound like a bastion of femininity.
Better than "On the building site"!