I don't necessarily mean GMO foods. However, sugar started to creep more and more into diets, even with the BB. It started with them because the mothers were enticed to work out of the home while the food companies came up with convenience foods to encourage and offset the mother being away. Now, we have tons of convenience foods, loaded with all kinds of not-so-healthy things.
Sugar, though, can definitely cause problems with fertility. For anyone who has suffered from infertility, one of the first things that the RE will tell you is to cut sugar out of your diet as much as possible.
Also, for women, especially those from the BB generation, who had progesterone problems to maintain pregnancies, certain hormones were given to them. We now know that some of those hormone-cocktails, even though they helped maintain the pregnancy, caused problems for them later in life. In fact, if you are pregnant nowadays, one of the questions the doctor asks is whether your mother was on these prescriptions because they may cause you problems with your fertility and pregnancy.
Thus, I think the answer to BB Catholics having/not having oodles of children is not necessarily cut-and-dry.