For those who defend that Vatican II is pastoral and not infallible, there seems to be two "proofs":
1. The Testimony of John XXIII at the opening of the Council:
“The salient point of this council is not, therefore, a discussion of one article or another of the fundamental doctrine of the Church which has repeatedly been taught by the Fathers and by ancient and modern theologians, and which is presumed to be well known and familiar to all. For this a council was not necessary. [...] The substance of the ancient doctrine of the Deposit of Faith is one thing, and the way in which it is presented is another. And it is the latter that must be taken into great consideration with patience if necessary, everything being measured in the forms and proportions of a magisterium which is predominantly pastoral in character.” (Opening Address, October 11, 1962; Walter M. Abbott, SJ, The Docuмents of Vatican II, p. 715)
2. The testimony of Paul VI:
“Taking into account conciliar practice and the pastoral purpose of the Council Vatican II, this Sacred Synod intends to issue in matters of Faith and Morals only the definitions it openly declares as such.” (p. 155)
Because none of the docuмents of Vatican II explicitly states the intention to define matters of Faith and Morals, one could conclude that Vatican II was pastoral, not infallible.
Also Paul VI himself declared the Council as non-infallible, in an audience one year later, when he stated:
“There are those who ask what authority, what theological qualification, the Council intended to give to its teachings, knowing that it avoided issuing solemn dogmatic definitions backed by the Church’s infallible teaching authority. The answer is known by those who remember the conciliar declaration of March 6, 1964, repeated on November 16, 1964. In view of the pastoral nature of the Council, it avoided proclaiming in an extraordinary manner any dogmas carrying the mark of infallibility.” (General Audience, December 1, 1966, published in the L’Osservatore Romano 1/21/1966)