you won't throw off the cycle, or the bacteria as long as you don't kill the bacteria (chlorinated water for instance). Basically what you have without the carbon is a biological filtration system, just like attaching a sponge to a powerhead. just rinse the medium out or squeeze it out every now and again to lighten the load of the workhorses (bacteria).
You'll be amazed the gunk that will come out just from dunking it in the water and swishing it around a little bit. I wouldn't scrub it and try to get it white again (or whatever color it was originally lol) or anything, but I usually dunk and squeeze it a couple times.