The main function of water changes is to remove organic wastes only
one of which is nitrates. There are many other biproducts of life, nitrates happen to be an easy one to test for in the home setting.
One thing fish excreet is a growth reducing hormone- this is what can stunt your fish's internal organs and development. This is not a good thing. The more time water and the wastes build up, the longer these things stay present.
Fish also absorb their nutrients/minerals through the water, as well as through food. Replacing old, used water with revitalized fresh water is the only way to put all of these micronutrients back into the water - just like we do for plants.
Also, fish have shown to be more active and energized when they are exposed to a freshwater influx.
In addition, by only topping off water every once in a while, rather than removing old wastes, leads to concentrated wastes and the buildup of potentially detrimental metals; especially if you have high concentrations of them in your tap water.
So yes, you still need to do water changes to keep happy, healthy fish.