Indeed it's confusing. If people can use the terms like "post-conciliar church" as if it's another church or say it's a new religion, meanwhile I'm getting scolded for making sure if I can pray with them. Also how is it they're not formal heretics, so praying with them in private - OK, but you can say a Pope or some clergyman is a heretic, yet not formal heretic, but praying with them - not OK. 
Pope question aside, you really need to read Abp. Lefebvre's "An Open Letter to Confused Catholics", "I Accuse the Council!" and "Against the Heretics" to understand WHY we refer to it as a "post-conciliar church"
As for heretical clergy vs laity, the clergy are public figures and speak publicly on matters of the Faith. When they are out there saying things like "all men can be saved" or
promoting sodomy or that divorced and remarried couples can receive Communion., then you have a serious problem with formal heresy.
Which is an entirely different problem than the poorly-catechised Novus Ordo friends you pray the rosary with who may or may not know that
Extra Ecclesia non Salus is a Dogma of the Church and that their "nice" Evangelical, Muslim or Jєωιѕн friends
will go to Hell if they do not convert.