Marulus,
(1) Augustine clearly believes that these infants are denied the beatific vision, and that this is a "condemnation."
(2) Beyond that, he clearly says that he doesn't know, but that it is "the lightest condemnation of all."
It's not just any condemnation, but a
damnatio cuм diabolo, condemnation or damnation with the devil, and it's a place of
ignis aeternus, eternal fire.
I believe that "condemnation" you speak of (it wasn't in the Latin I looked at posted earlier), actually refers to
poena, the actual suffering or affliction or torment part, and not just the condemnation. But I'd have to find more of his Latin to be sure.
But it's important whether the "mildest" refers to the condemnation or the punishment. These are potentially two distinct things. Let's say that two of my children are involved in the same act of mischief. I ground them both for 3 weeks. So they get the same punishment. But I know that the older one was exerting his influence over the younger one, so I am less upset with the younger one than with the older one. Or, another analogy, two people cause $2,000 damage to my car. One of them did so accidentally, the other vandalized the car out of spite. Both owe me $2,000 in damages, and that would correspond to the
poena, the penalty, but I'm not upset with the individual who did it accidentally, perhaps even feel sorry for him, but I'm angry at the one who did it deliberately. So there's a distinction between guilt and punishment.
This distinction, BTW, is also the key to understanding Pope Pius IX's famous "invincible ignorance" passages, where he states that those not guilty of actual sin would not be afflicted with punishments, in Latin, the
poenis, or penalties, but this does not necessarily mean they will be rewarded with Heaven.
St. Gregory nαzιanzen, in rejecting Baptism of Desire, states that there are some who are not bad enough to be punished but not good enough to be glorified. Our Lord taught that those who believe and are baptized will be saved, but that those who do not believe will be condemned. This leaves a middle ground of those who believe but are not baptized, where they fell into neither category.