Because the modest families have 10 children while the immodest ones have 5. Eventually the laity will become modest again, by process of elimination.
Be careful there.... 5 children is well within range of assuming a family isn't using NFP, much less (mortally sinful) artificial birth control. It is within range of "normal family size" at least for some towards the back of the curve.
Family size is like a normal distribution curve (like IQ) -- there's the low back of the hump, the peak in the middle, and then it slopes down again on the other side. Right now the "peak" is at 2 children whereas it should be peaking around 7 or so.
I have a serious problem with those who put "10 or 12 children" families on some kind of pedestal, as if they are more Catholic, more serious, more Trad, or more holy than my family with 7 children. Not everyone has the same fertility, not everyone marries at 18 or 20, and not everyone breastfeeds their children (which is best for them).
The Church doesn't judge by numbers -- neither should we.
I think some of these "10 to 12 children" families that people look up to are just sinning in a different way -- they are proud, and feel they have "out-Tradded" everyone. Basically their mentality is that of the Pharisees. They fulfill the letter of the law better than anyone around them, so they're "the best" they think.