The general answer is some kind of leggings, tights, or knee length shorts or culottes with a half-calf or longer skirt or dress, season and activity depending. Having lived with Old Order Amish of the very conservative variety, that is exactly what they do. I discovered that Amish dress with apron and pantaloon type pants and slip underneath is cooler than shorts in summer and the woolen long version with wool stockings is warmer in winter than just pants or pants over t-shirt knit long underwear. Try it out, ladies!
If your skirts do happen to fly up, there’s nothing to be seen. If working out or the like, Amish ladies don’t get wedgies!