you should just be able to punch it out, if I remember correctly. if you don't mind my asking, why are you taking it all the way off? I always just let that one hang down when I take the tank out.
I painted mine with them on. the finish was not perfect, but it looks good from a few feet away, which is all I cared about for such a minor detail. I would think you could just tap it back in, but I'll go out and check mine later tonight and get back with you.