Actually, there is alot of confusion here regarding several points:
1. Everyone here HAS to agree that when an unbaptized infant dies, they go to hell in SOME SENSE, because ALL the scholastic theologians place the Limbo of the infants IN HELL.
2. The "speculation" on infant damnation and limbo was never a speculation as to PRINCIPLE (That unbaptized babies are deprived of the vision of God and are punished in hell), but as to DETAIL. I.E. what do the pains of the damned infant consist of? Augustine said Fire, but to a lesser extent, Abelard said spiritual suffering from the lack of the vision of God.
3. St. Albert the great was the FIRST to coin the term Limbo in reference to the fate of Infants, and St. Thomas was the first to teach that it consisted of zero pain, and in fact natural happiness. This is 800 years AFTER St. Augustine.
4. If St. Augustine was in error, why did nobody before Aquinas come to his conclusions?
5. The DOGMA of the Church, as set forth by the ecuмenical councils of Lyons, Florence and Trent, Specify that those who die in original sin alone are in Hell, where they are punished, but with a different punishment from those who die in mortal sin. There is room to discuss both the type and degree of punishment if we go by the language of this canon alone.
a. Key question: How many types of people die in mortal sin alone? Generally, one: Those who do not have the use of reason; specifically, two. Infants, and the mentally handicapped.
6. The Council of Trent has authoritatively declared in the fourth paragraph of the fifth session, that the words of our Lord "unless a man be born of water and the Holy spirit, he cannot enter the kingdom of heaven" apply to infants.
"... For, by reason of this rule of faith, from a tradition of the apostles,
even infants, who could not as yet commit any sin of themselves, are for this cause truly baptized for the remission of sins, that in them that may be cleansed away by regeneration, which they have contracted by generation. For, unless a man be born again of water and the Holy Ghost, he
cannot enter into the kingdom of God."See here: The decrees on original sin, paragraph four
http://history.hanover.edu/texts/trent/ct05.html7. The Catechism of the Council of Trent, Promulgated by a Pope and a Saint, Pius V, is the AUTHENTIC interpreter of the council of Trent. It states without any lack of clarity that those who are not born again, are born to eternal misery and destruction. See Here: Key word search "misery"
http://www.catholicapologetics.info/thechurch/catechism/Holy7Sacraments-Baptism.shtmlNecessity of Baptism
"If the knowledge of what has been hitherto explained be, as it is, of highest importance to the faithful, it is no less important to them to learn that the law of Baptism, as established by our Lord, extends to all, so that unless they are regenerated to God through the grace of Baptism, be their parents Christians or infidels, they are born to eternal misery and destruction. Pastors, therefore, should often explain these words of the Gospel: Unless a man be born again of water and the Holy Ghost, he cannot enter into the kingdom of God."
Baptism Of Infants Should Not Be Delayed
"The faithful are earnestly to be exhorted to take care that their children be brought to the church, as soon as it can be done with safety, to receive solemn Baptism. Since
infant children have no other means of salvation except Baptism, we may easily understand how grievously those persons sin who permit them to remain without the grace of the Sacrament longer than necessity may require, particularly at an age so tender as to be exposed to numberless dangers of death."
I could go on...We haven't even seen the quotes of Sts. Augustine, Fulgentius, Prosper and Caesarius, Gregory I, etc....