I think it depends on the situation. For example, I think it would be fine, perhaps even preferable, if an unmarried male relative (uncle, cousin, older brother, etc.) wanted to adopt a recently orphaned child. In this situation, the child remains with his family, and would likely not be devoid of female authority figures, even if the adoptive father is single.
On the other hand, if a single man (or woman, for that matter) simply went to an adoption agency or orphanage looking to adopt a child, it would be far from ideal, and likely shouldn't be permitted, especially if there are married couples also seeking to adopt.