That depends...
Nowadays, it's likely that your municipal water supplier is adding chloramine, not chlorine. Chloramine doesn't "gas off" in any reasonable period of time.
You can contact your local water department for clarification on what they add, but you're probably better off just adding a dechloinator.