Don't lose your Faith because of faithless people. Trust in Our Lord Who is always faithful, and promises He will never leave us or forsake us. The SSPX is neither the Faith nor the Church. It is sinful to give scandal, but it is also sin to take scandal. St. Paul says to be angry but sin not. Don't allow the sun to go down on your wrath. Pray often to forgive this man.
Also, know your experience with the SSPX, the priests, or those to whom the priests give power are by no means one of a kind. I was booted out of an SSPX chapel in 2014 because I attended a few private Masses said by an expelled SSPX priest whose family happen to be long-time family friends. I also got blamed by a group of women who more or less rule the priest for writing a rude letter to Fr. Rostand, then the US District Superior. The rumor was based upon my having been seen placing an envelope addressed to Fr. Rostand in the collection basket. The claim was ludicrous because the letter contained a request and stipend for Gregorian Masses for my aunt. Someone from the chapel did indeed have a dispute with Fr. Rostand, but since she is wealthy, socially and politically prominent, the priest or someone higher up didn't want the "scandal" and loss of income it would cause, should she be outed. How much easier to blame a relative newcomer, a convert from Protestantism, unknown, and "poor." (I put poor in quotes as nearly everyone in the chapel was well-off, owners of one or more homes, new vehicles, living in up-scale neighborhoods. By contrast, I live in a rented room three flights above a store in a not so great neighborhood. I have a 14 year old car with dents, but it runs well! I lack for nothing really needed, so I don't consider myself in poverty.)
Both matters came to a head in October of 2014 when I was told by the head Sacristan that I should not come to Mass because I'd caused scandal to come on the chapel. I've stayed home ever since, praying the Rosary, keeping up with the readings, getting sermons on line, keeping the Faith as best I can by myself. There is no more Resistance Mass in my area, and no other choice but the novus ordo.
For about three months, I was very depressed. On Sunday I'd drive to some secluded spot to pray, but end up crying, so I just went home.
Eventually, I realized I'd taken scandal by allowing unjust people to separate me from God. I confessed this to OurLord, made as close as I could of an act of perfect contrition, and got on with it. I will continue until and if Our Lady opens a door to a normal Catholic life.
OP, you've exhausted your resources. Let go of the man and his money. Forgive him, offer your loss up, and get on with being Catholic. Sometimes it is necessary to literally "shake the dust off your sandals" and walk away.