In most cases, I would agree to always clean the gravel when doing a water change. The only exception is if you have a heavily planted tank and want to keep the 'mulm' for the plants to use as food. If that is the case, you always want to test the water regularly for any spikes in ammonia or nitrite, because too much mulm is a bad bad thing!