It could be the gravel but my first course of action would be to double check all of the water parameters and make sure that it isn't spreading onto their fins or other fish - which would indicate a bacterial/fungal infection.
Something to keep in mind is that the gravel really should be very clean in their tank. Since their mouths are in almost constant contact with the gravel, they can get infections pretty quick if it's dirty.
I'd add some melafix or antibiotics to help grow their whiskers out w/o getting an infection. If you can put them in a hospital tank, that'd be great.
I"m not sure what size tank you have, but if you have room, I suggest getting at the minimum one more cory. They like to be in groups and 3 is really the smallest group I'd do
It will make them more comfortable and probably a bit more active
Good luck!