It's interesting that so many posters have presumed the OP is female when, according to the OPs profile, he's a he.
I think Simple Man gave the perfect answer.
She's clearly female from the way she talks. And men don't blush when someone waves at them across the street.
OP, being attracted to someone other than your spouse is a temptation to sin. Not a sin itself, so you do not sin as long as you avoid it.
You should quit your job if you can and stay at home with the children because that is the chief role of a mother in Catholicism.