Wives aren't to leave home without the husband's permission - it says that in the Roman Catechism.
Normally a trusted wife would have blanket permission and the issue need not come up.
I think a man could be justified before God in restraining his wife like that, but the odds of repercussions in our unnatural feminist society are very high.