What if the priest asks you to tell you the name of the priest? Can you refuse?
It depends, but no, you cannot refuse. You can say that you prefer not to name him, but if the priest insists (which he shouldn't), you have to tell him, otherwise you are not obligated to name him, more like your first obligation is to not name him.
But it depends on a number of things including what was said, was it lies, truth, said out of anger, who you said it to, who all heard it, is the priest / what he says well known (aka Fr Ripperger for example) and on and on. No one expects you to announce such details on a message board so don't.
When you go to confession, put out of your mind that you are confessing to a priest, imagine instead that behind the screen you are confessing God Himself, to Jesus Christ, who could ask some very prying and uncomfortable questions if you give insufficient details. Yet you are expected to practice the 3 "B"s - be Brief, be Blunt, Begone.