The fact that Pope John XXIII referred to it as a "Pastoral Council" is irrelevant. He also referred to it as an Ecuмenical Council many times, including most importantly in the opening speech of the Council. Ignoring that and just deciding to ignore an Ecuмenical Council because you dislike it is no different than what Old Catholics did. You have to actually justify why an Ecuмenical Council called by a valid Pope is somehow not a valid Council.
Under your logic, would you consider the Lateran Council of 649 an Ecuмenical Council? A True, Valid Pope intended it to be an Ecuмenical Council, and it was referred to as such as the start.
https://en.wikipedia.org/wiki/Lateran_Council_of_649Remember what makes a Council Ecuмenical is Papal Ratification after the Council, not words said before or during.
Pope Paul VI stated that it's not infallible. Vatican 2 was a heretical robbers council, there have been many in Church history. Full stop.