How often are you near the tank? If you're only at the tank when it's feeding or cleaning time, these fish are going to associate disturbances in their tank with this big human standing outside of it. I've noticed that my fish have become less skiddish since I started reading the newspaper next to the tank in the mornings, or (sometimes reading a good book at night). I know there isn't much thought going through their little brains, but they do seem to be curious as what's outside their little world.