Despite claims to the contrary, St. Ambrose did not teach BoD. St. Augustine taught it during his youth but later emphatically rejected it. Those two there, "Augustine and Ambrose", have wrongly been used in all subsequent theorizing about BoD as the "authority" for BoD.
This is where St. Ambrose's oration of Valentinian has been completely misunderstood. He believed that Valentinian, like the martyrs, could obtain some of the grace of Baptism, the washing part, but said in the same speech that "not even the martyrs are sealed", even though they are "washed". But the seal was considered essential by the Church Fathers for entry into the Kingdom.
So later, Pope Innocent II opines in favor of BoD "on the authority of Augustine and Ambrose", and St. Bernard said that he'd rather be wrong with Augustine than right on his own. These two opinions actually turned the tide of the debate in favor of BoD, but they were both mistaken. Not only is it absurd to ignore the 5-6 Fathers who rejected BoD (what were they, chopped liver? ... did Augustine and Ambrose have some kind of higher authority than these others?), but both Innocent and St. Bernard were simply WRONG that these two taught BoD. St. Augustined did ... for a while ... but then later rejected it. St. Ambrose never taught BoD.
Interestingly, even Catholicism.org couldn't understand St. Ambrose, chalked it up to either a contradiction or to some unknown factor. Well, the answer is right there in the text where St. Ambrose says that not even martyrs are sealed.
If you take St. Augustine and St. Ambrose out of the mix, then you end up with ZERO Patristic support for BoD. So, not only do you not have unanimous Patristic consensus in its favor, but you have NEAR-unanimous Patristic REJECTION of BoD, and it's 100% unanimous of you take the opinion of St. Augustine later in his life rather than his former opinion.