Chapter 93. Both the First and the Second Deaths are the Consequence of Sin. Punishment is Proportioned to Guilt.

And neither the first death, which takes place when the soul is compelled to leave the body, nor the second death, which takes place when the soul is not permitted to leave the suffering body, would have been inflicted on man had no one sinned. And, of course, the mildest punishment of all will fall upon those who have added no actual sin, to the original sin they brought with them; and as for the rest who have added such actual sins, the punishment of each will be the more tolerable in the next world, according as his iniquity has been less in this world.

CHURCH FATHERS: Handbook on Faith, Hope and Love (St. Augustine) (newadvent.org)

But I do not say that children who die without the baptism of Christ will undergo such grievous punishment that it were better for them never to have been born, since our Lord did not say these words of any sinner you please, but only of the most base and ungodly. If we consider what He said about the Sodomites, which certainly He did not mean of them only that it will be more tolerable for one than for another in the day of judgment, 2 who can doubt that nonbaptized infants, having only original sin and no burden of personal sins, will suffer the lightest condemnation of all? I cannot define the amount and kind of their punishment, but I dare not say it were better for them never to have existed than to exist there. But you, also, who contend they are, as it were, free of any condemnation, do not wish to think about the condemnation by which you punish them by estranging from the life of God and from the kingdom of God so many images of God, and by separating them from the pious parents you so eloquently urge to procreate them. They suffer these separations unjustly, if they have no sin at all; or if justly, then they have original sin.

The Fathers Of The Church A New Translation Volume 35 Saint Augustine Against Julian : Roy Joseph Deferrari : Free Download, Borrow, and Streaming : Internet Archive

who can doubt that nonbaptized infants, having only original sin and no burden of personal sins, will suffer the lightest condemnation of all? I cannot define the amount and kind of their punishment, but I dare not say it were better for them never to have existed than to exist there. But you, also, who contend they are, as it were, free of any condemnation, do not wish to think about the condemnation by which you punish them by estranging from the life of God and from the kingdom of God

He distinguishes his position from those who say the infants are "free of any condemnation" and proceeds to elaborate that they admit a single condemnation, the deprivation of the Beatific Vision that is.

Clearly, Augustine believed infants suffered some kind of punishment besides the "punishment" (improperly so called because not receiving an inheritance is not a punishment) of deprivation of Heaven his opponents held.

FIFTH SESSION

2. If any one asserts, that the prevarication of Adam injured himself alone, and not his
posterity; and that the holiness and justice, received of God, which he lost, he lost for
himself alone, and not for us also; or that he, being defiled by the sin of disobedience, has
only transfused death, and pains of the body, into the whole human race, but not sin also,
which is the death of the soul; let him be anathema:--whereas he contradicts the apostle
who says; By one man sin entered into the world, and by sin death, and so death passed
upon all men, in whom all have sinned.

SIXTH SESSION, CHAPTER I.


On the Inability of Nature and of the Law to justify man.

The holy Synod declares first, that, for the correct and sound understanding of the
doctrine of Justification, it is necessary [Page 31] that each one recognise and confess,
that, whereas all men had lost their innocence in the prevarication of Adam-having
become unclean, and, as the apostle says, by nature children of wrath . . .

I'm perfectly aware of the CE article on Limbo, and on baptism of desire, etc.

As I said: where is the quote from St. Augustine saying infants dying without baptism are subject to "torments" and the "fires of hell"? Still waiting. As of now, it's still in the "myth" category.

Abelard was also a heretic on several issues.  It's my understanding that Original Sin is an actual sin, not just a condition or status.  I think that idea is where the severity of the St. Augustine view springs forth.

You're not a very clear thinker.  We're distinguishing between the non-suffering (in fact, perfect happiness state) of Limbo vs. the suffering state.  As the EENS definitions tell us, each suffers in proportion to their sins.  What was under dispute was whether Original Sin alone led to suffering in Hell period, and not the degree of suffering.  St. Augustine held that their suffering was mild, but that they suffered nevertheless, i.e. experienced a positive punishment.  St. Thomas stated that not only do they not suffer, but they experience perfect natural happiness.  It's all laid out quite clearly in the CE article.  We're not arguing about the degree of their suffering but about whether they suffer at all.