I'm gonna just go at this logically, so if I say something heretical, let me know, I've never thought about it before.
I would say it 'could' be theologically possible, as Bellarmine would say, but only because it has not been directly condemned.
I'm going to compare such a proposition to that which the earliest catholics, the apostles, would have faced. It is quite likely that in their early ministry, they were quite scattered, with only a handful of lay people, and similarly a lack of clergy.
Irenaeus states the apostles appointed Linus to Bishop of Rome, and we can imagine the difficulty for them to gather, decided, etc.
I would say if there were but a few scattered clergy remaining, they would be obliged to seek, meet and elect another pontiff, as that would be there duty as the remaining visible church.