Yes, you should completely avoid the SSPX if your priest has ANY modernism or liberal/modernist leanings.
I'm not sure I can agree with that. So, the PRIEST could have Modernist leanings, but perhaps those don't manifest from the pulpit. Also, it depends on many other prudential considerations ... such as whether you have young children or others in your familiy who might be influenced towards Modernism. So, if it were just I, my attitude would be ... "that's his problem, between him and God, but I desperately need the Sacraments and have a Canonical right to receive them, and I won't be deprived by some Modernist". And beyond that, even, you have to weigh how much harm would be done to your soul even with some relatively-subtle Modernist influences (Father Paul Robinson is an outlier, since there's nothing subtle about his nonsense.)
Bishop Williamson used to say that we are ALL infected by Modernist / Liberal attitudes, to one extent or another, where it's just a question of how much, and so there are likely very few priests who are not tainted by ANY Modernist tendences.
Also, in the 1940s and 1950s, Modernism and Liberalism were already running rampant back then, but unless you could point to some very concrete errors being taught, you would not be excused from making your Sundya obligation.
I don't believe there are any absolutes here, since, contrary to the dogmatic "una cuм" types, I don't believe that there's anything intrinsically evil about attending any given Traditional Catholic chapel (provided there are valid Orderrs and they at least profess Catholicism). Whatever evils there might be there are accidental to the Mass, and, coming full circle, "the priest's problem, for which he, not I, will have to answer to God for".
Prudence is that queen of virtues ... and everyone's circuмstances can vary so wildly that i don't believe there are absolute applications of principles to be found in a one size fits all manner. One person might be unable to stay in a state of grace for more than a couple days at a time and need regular access to Confession and Holy Communion. To paraphrase Our Lord, better to go to your judgment with a couple of subtle Modernist-leaning tendencies to go, completely orthodox and Traditional, into the fires of Hell. Another person might have a weak mind or weak faith and be easily scandalized by a Modernist priests ... others, who are well grounded in the faith, are the first ones to spot the Modernism and it'll run off of him like water off a duck's back. And, even if you can stay in a state of grace, not receiving Holy Communion for long period of time can cause the soul to wither up and dry out ... and could over longer periods of time ultimately lead to a moral fall.
I am of the opinion that the Modernistm / heresy would have to be over-the-top pernicious, and so bad that you're not able set up protections against it and / or to simply ignore it ... before it would justify staying away from the Sacraments and Mass on at least Sundays and Holy Days. Similarly to purity, some people are susceptible to sin with very little, while another could would around on nudist beach and almost not even notice it were it not for the surprising nature of it and of finding oneself there. Similarly, some might be impervious to error and Modernism, while others can be influenced too easily and their weak faith shaken.