Is it reely bad for a finger leather coral to be touched with my hand will it die?

I've got a leather finger coral in my saltwater fish tank that has been up and running for 3 years.I just bought the coral and I handled it twice with bare hands to put it where I wanted it. It has like 3 spots of grey on it now is it going to re color is it going to die I'm really Woried plz help!