The biggest problem with widespread annulments: it means that Traditional Catholic men & women can have multiple living sex partners, just like in the World. You could run into someone at the store who you had the most intimate of experiences with, who isn't your current spouse.
The only way a man/woman should "experience" more than 1 spouse is when a spouse dies. But if you think about it, that's quite different. You're not going to run into a deceased spouse at the store, children won't blame you for moving on from their mother/father (since you didn't kill your spouse), and there isn't any possibility of jealousy, since your past spouse(s) are deceased. Your current spouse won't be jealous of a deceased person, and vice-versa. No drama. You can still look your kids in the eye and say you love them, especially because they are part "your previous spouse" who is now deceased, and probably very similar to them.
When you move on from a living spouse -- be it annulment, divorce, whatever -- you are rejecting that person, and by extension your children who are 50% that person.
It amazes me how shallow people can be. My wife's parents are divorced. How can my father-in-law claim to really love his daughter, when she is SO MUCH like the woman he spurned? At best, he could claim that he loves her, but he doesn't like her. They're both Catholic, musically inclined, homebody, risk-averse, stable, sober, responsible, down-to-earth, family-oriented, frugal, rule-following accountants.