Two issues to address - first up is your Ph.
Most Tetras are sensitive to Ph changes, and everytime you do a water change they are going to be in trouble. You need stability in there, and for these fish a low Ph is better than a high one - (6.2 was fine). As long as the introduction of your tap water at 7.8 is done slowly, the tank probably wont go much over half a degree higher during a 20% change, which the fish should be ok with if it’s done gently.
As that rock and the shells are making your Ph move around - get rid of them. Let the tank settle at 6.2 if that’s what it does - the Tetras will be fine. And next time you buy fish - ask the shop what their Ph is - and introduce the new fish gently if there’s a difference of over half a point.
Getting your Ph stable is as important as what it settles at - but like I said - lower is better for Tetras - so the rock and shells go.
Algae ..... gotta love it - lol. Two things make algae grow - light and food. Green algae is usualy the result of too much sunlight hitting the tank - and it’s green because it feeds off the sunlight just like any other plant. Brown algae is more common in a tank where there is too much food in the water - dissolved food you can’t see with the naked eye. So - as you have brown algae - cut down on the feeding and keep the tank clean.
With regard to cleaning that tank of yours the set-up is a nightmare. The gravel will hold food - all the nooks and crevices around and between the rocks - the folds of the leaves on those plants ....... it’s just one big food trap.
As you’re getting rid of the rocks, lets concentrate on the rest of the tank. Gravel can be cleaned yes, but by its nature it has so many places a food particle can drop into, you’ll never win the battle. You could try using sand, or go bare bottom - either of which will see a marked reduction in uneaten food retention. Leave enough space inbetween ornaments and plants to get your cleaning suction device (or whatever you use) in there. Try to eliminate “no-go” areas.
The plants are easier - once a month take them out of the tank completely, and wash them off in clean water using a brush. There’s no way you’ll stop that type of plastic plant from accumulating detritus, so accept you have to deal with it properly.
Yes - your water is killing your fish - but it’s doing so for predictable reasons - and it’s not whats coming out of the tap that’s doing it - it’s the tank set-up.
Time for a re-design in there ...........