The fruits are not to be seen/measured according to all people, because the sspx (just like the catholic church) is made up of flawed, sinful individuals.  The fruits are to be seen in exemplary individuals, but most importantly, in the Truth/doctrine.  That is, that priests/families are willing to sacrifice x, y and z to protect, (attempt to) build Tradition, build schools, build chapels/convents, in hopes that Tradition will survive.  The struggle/desire/work is proof that Tradition is worth keeping.

In other words, there was a story of a protestant who was serious about catholicism.  He visited Rome in the mid 1800s, admist all sorts of Vatican scandals, etc.  He returned and shocked his friends by his decision to convert.  He said, "If the Catholic Church can survive all of the turmoil, scandals, and lukewarm individuals I met in Rome, then surely it only exists because God is keeping it going.  Therefore, it must be of Divine origin."

If Traditional Catholics all believed in EENS, none of us would have any doubt but that that Vatican II was heretical and that the Conciliar papal claimants have been Anti-Popes.  There would be no more needing to stretch and read into this ambiguous statement or that ambiguous statement by one or another of the Anti-Popes.  It would be crystal clear to the eyes of faith, and the only reason everyone doesn't see it is due to weak faith.