CathInfo.com

Let me get my ladislaus signal

Yes, St. Augustine did at least in the beginning believe in BoB.  During his early speculation regarding BoD, he did in fact state that he accepted the reasoning of St. Cyprian on the matter of BoB.  I think I recall seeing, however, that he later rejected the argument made from the example of the Good Thief, and so his final position remained unclear.  If you examine what St. Cyprian actually argued, though, he believed that those who had BoB actually received the Sacrament of Baptism.  Some, such as the Dimonds, have said that this was an error.  But I saw (and once quoted here) passages where St. Cyprian actually spoke of the angels pronouncing the words of Baptism over the dying martyr and where his blood served as a replacement for the water of Baptism.  So my belief is that St. Cyprian literally meant that BoB was the Sacrament, just an extraordinary manner of receiving it, where the martyr's blood would be valid matter and the angels supplied the from.  Since St. Augustine followed St. Cyprian on BoB, it may be conjectured that he held a similar view.  Also, St. Ambrose was a teacher of St. Augustine as well, and St. Ambrose referred to martyrs as being "washed but not crowned" and stated that not even martyrs could be crowned (and therefore enter the Kingdom of Heaven), so that this BoB would consist of a "washing" and yet would not suffice for entry into the Kingdom of Heaven.

Of what does this washing consist.  I believe that it washes away the temporal punishment due to sin, the punitive aspect of sin, but not necessarily the guilt of sin, as St. Augustine said there can be no remission of (the guilt of) sin without the Sacrament.  But the punishment could be washed away.  Thus, with martyrs, it's commonly held that they go straight to Heaven without any time in Purgatory because their temporal punishment is wiped away.  Thus, I speculate, that someone who were to die an unbaptized martyr (barring St. Cyprian's angels scenario), all his temporal punishment due to sin would be wiped, the actually suffering due to sin, the poena of sin, and so he would end up in Limbo along with the infants who have no punishment due to sin.