I'm bored and feeling helpful so here is a really long post. Hope it helps point you in some direction as well as get you inspired.
This probably sounds terribly geeky, but I love the whole idea of keeping a small piece of life in my home. There is the thrill of keeping a functioning biological system intact, as the science of the whole thing appeals to my linear-logical side. Picking and choosing the proper methods of maintaining the bioligical functions such as bacteria, plant needs, fish needs, choosing the best substrates, filtration systems, lighting, etc. All of these make a little bit of a scientist out of the hobbyist.
Then there is the creativity and artistry behind it. That is what appeals to the more abstract / artsy side of the personality. These things involve choosing what sort of a tank you will have, what plants, fish, decorations, etc. you will use to make the tank beautiful to look at.
Thus when you start having serious success in this hobby, you get to feel like you have a certain geeky coolness about your abilities to make this thriving little piece of life in your home. You also get to stare into it and watch the fishies do their thing.
Okay, so long post. (I'm known for them) I'll just say that the best way to get started here is to read all you can about everything. Water chemistry, fish types, plant requirements, filtration systems, etc. Then you decide how large of a tank you can afford. With this comes the plan for the types of fish and plants you'll want to keep. After that, the trick is to apply your knowledge to how you will go about making the tank that you want to have.
They say to always buy the largest tank you can afford. This is true to a point; but keep in mind that often we can afford the tank but then once we figure in the money for a stand, sufficient lighting (especially for keeping moderate to high light plants) proper amounts of substrate, rocks, driftwood, etc we find that we're not as wealthy as we thought
Plus larger tanks require more involved maintenance. Though smaller tanks kind be a little harder to manage due to the fact that they are less stable as far as water chemistry. They also limit our ability to have lots of fishies.
It's a general rule of thumb that long tanks are better than tall tanks. (This is arguable in cases like angelfish or discus, which are often longer than they are wide.) The long tank is best for most fish because, well....just look at a fish. They're long and that's the way that they swim: lengthwise.
If you plan to keep plants, be sure you can afford quality fluorescent type lighting. (Lighting is something to read...read...read about.) Usually lights that provide 2-4 watts per gallon can grow pretty much any easy to care for plant. Figure the price of a good lighting setup into your overall cost budget.
You'll need a proper stand. (No, the milk crates won't due here!) For ten or fifteen gallon tanks, you can probably get away with using a sturdy table. Anything larger will likely require a proper stand, and these can be more costly than you think. (But much better than the notion of a collapsed dresser and a floor full of water and dead fishies!) A worthwhile investment.
Substrate is a place of many options like the lighting factor. This is where you will exercise your knowledge and creativity to pick the one you like best. (Or you'll create your own mixtures, etc.) Then again, many people are happy with that hot pink or purple gravel. (Yuck!) But to each their own I suppose. Just know that most fish will appreciate a darker, more natural substrate.
Then there are rocks, decorations, etc. All depending on the effect you seek.
If you're choosing a tropical tank (as opposed to a cold water tank - goldish, etc.) you will need a quality heater. I personally recommend Rena Cal Basics, Ebo-Jagers, and Visi-Therms. Tropical fishies like to be in temps between 72 and 80 degrees give or take. AVOID budget cheapy heaters. I've heard enough horror stories to know how worthwhile a quality heater can be.
Learn about cycling the tank. That is a most important aspect of getting started, and will help you avoid "new tank syndrome."
...And of course the fish! Educate yourself on the different species. How they co-exist or do not co-exist. What sorts of water chemistry they prefer. How big they get and if they will live happily in the size tank you have set up.
Here is how I set up the tank I have now:
I decided I wanted to do a loose interpretation of the SE Asian blackwater biotope. What I did was read everything that I could about SE Asian water chemistry, native species, etc. I also wanted a low-maintenance sort of a tank because I'm slightly lazy.
The answer was then to pick the hardiest, easiest species of plants and fish to care for, and put them into a tank that would most likely imitate their natural environment. (with modifications of course) I chose danios, gouramis, bettas, loaches and rasboras. For plants I chose java fern, java moss, and crypts. The substrate was mainly flourite and river rock, all in dark red hues to most closely resemble the iron-rich substrates of SE Asia. I bought some Malaysian driftwood. I chose a canister filter so that I could use peat filtration in order to make my water soft and acidic, as well as give it the "blackwater" effect.
I started with a ten because it was already in my posession from when I was a kid. I maintained that setup for close to three years until last week when I finally upgraded it to a twenty gallon long. I couldn't be happier.
Of course, you will choose any type of tank you want to keep. That is the real fun of this hobby. I hope this got you thinking creatively.