After a while, wine residue/vapor must find it's way into the inside of glass stoppers where mold eventually forms.

On the glass stoppers I have, the plastic end is held on so tightly that prying it off will damage it. Pliers to grip it wouldn't work because they will compress the plastic preventing it from sliding over the glass lip.

The plastic needs to be pried off from all sides simultaneously to prevent damage. One way of doing this is to wrap fishing line around in the gap where you would normally try to pry the plastic off. After 40-50 wraps of moderate tension, It took another 60 high tension wraps to pop the plastic off.

The idea is to get the fishing line to build up in the gap, but tightly wrapping it forces the line to squeeze beside itself like driving a wedge in there on all sides. The tension of each turn really adds up until the plastic cap is pushed off. Its best to try to not criss cross the wraps or they can't easily slip down between the other wraps.