We never miss a chance to provide religious gifts for baby:
a Miraculous Medal to pin near the baby's bed, out of reach of course.
baby Rosary as an earlier poster mentioned
Catholic board books, baby's prayer book, true story of Christmas, Easter, etc.
Holy card or book about the baby's Saint name (if applicable)
Catholic music for babies
child's Nativity set
book for the parents on raising a Catholic child