I remember once reading from a private revelation and one of the things revealed was that one of the main reasons a man was saved was because he listened to his priest's advice and did not have marital relations with his wife when she was pregnant. It didn't say it was a sin to have relations while pregnant but that it pleases God when you don't kind of like fasting. I wish I could remember the source I read this from so I could provide a link.
Sure, abstinence can certainly please God and be meritorious ... unless it doesn't please God. So much of this depends on the circuмstances. So, for instance, you often find the "pious" type who becomes reluctant to render the debt in the interests of a higher purity and spiritual pursuits. But this leaves the other party feeling put out, based on one or another of the secondary ends that they feel the need to have satisfied. In that case, it would be blameworthy rather than meritorious. Out of charity, the more pleasing thing to God would be to NOT abstain in that case. So many people have been tainted with this idea that there's something inherently incompatible with sanctity in pleasure. God would not have associated pleasure with physical relations if that were the case.
Reminds me of a story about St. Francis of Assisi. He had been abstaining from meat for several years. At one point he was invited to eat at someone's home as a guest, where the person served meat. St. Francis ate the meat without a second thought. One of his fellow monks asked him whether or not it made him feel bad to have eaten meat after so long. He said that he hadn't even given it a second thought because not eating would have been rude to his host. That's true humility and true charity ... not the person who wants to abstain from physical relations with a spouse even against that other person's wishes.
Sin ultimately resides in the will and not in the flesh. People need to remember that.
It's also why a lot of people get so worked up about sins of the flesh vs. other types of sins ... because they're ashamed of the physical acts. But those same acts are NOT sinful within the context of marriage. It's not the act itself that's sinful or shameful but the fact that one decided in his WILL to do something against the law of God. That is the essence of sin. In fact, sometimes sins of the flesh come from mere weakness, and there are many other forms of sin against the laws of God that entail greater malice or lack of charity but for which people have little shame.
So let's say that someone in a moment of weakness slips up and briefly consents to an impure thought. Now, another person decides he doesn't feel like going to Mass on Sunday for no other reason than he doesn't feel like it or would prefer to watch football. Which is the greater sin? Obviously the latter. Yet people are more ashamed of the prior than of the latter ... whereas it should be the other way around.