If you have rocks in your tank, I would take one out (a decent-sized one), tie the veggie to it, then sink it back into the tank. They also make veggie clips that are weighted, but I'm not a fan of the way they look.
I would use split shots or weights from fishing tackle in your tank...if I'm not mistaken, most of those are made out of lead, which can poinson your fish.