It's a good point about the man having no rights to fidelity, since he and his concubine never made a commitment in the first place, there's no contract or promise to be broken.
However, how is a common-law marriage any different from 95% of "marriages" out there in America today? Most mainstream Protestants believe in divorce, and none of them have their marriages witnessed by a Catholic priest. So they're all technically shacking up.
But does the Church see it that way? What does the Church say about non-Catholic unions where both spouses were never Catholic? Doesn't she say those marriages are valid? Remember, a justice of the peace, some random protestant "minister" or a ceremony that never took place -- it's all the same in terms of actual legitimacy. None of them are a Catholic priest representative of the Church. You're basically talking about a Natural Law marriage, one man, one woman. 100% natural, but no supernature or grace involved.
Now one-night stands or "hookups" are a completely different animal. Or casual "flings" lasting days or weeks. But these cases where a couple shacks up together for years, even has children together -- that's a "common law marriage", and many secular laws take this into account.
But the idea that "It's not a supernatural marriage? (two baptized Catholics married officially by the Catholic Church) No big deal then. Carry on with your infidelity, deceit, cuckolding, and adultery" doesn't sound right. Yes, Catholics have more to lose, more to be destroyed by an unfaithful spouse -- but if we are not to care at all about less-than-supernatural unions, then at least 95% of the marriages in America are not to be valued at all.
Heck, most worldlings even look down on (as in "tsk tsk") a boyfriend cheating on his girlfriend, or vice-versa. And yes, these boyfriend/girlfriends are living like they're married. But their relationship would fit into the "short term fling" category -- and yet most worldlings have no respect for someone who cheats on their boyfriend/girlfriend -- i.e., without breaking up with them first and becoming "single" again.