The piece your talking about houses some bacteria too. Most filters(HOB) have something called a bio-wheel or something of that nature where a lot of the bacteria collect. If your changing the filter cartridge in a HOB you're not really removing that much bacteria, or not enough to put you into a mini-cycle IMO. The bacteria live everywhere in your tank so you plants, ornaments, gravel, random pieces in your filter (like your removable one). Now if you change your HOB unit out with another one then you should be careful.