Coins are fascinating little things and collecting them you can't avoid learning about metals and history, including the HISTORY OF MONEY. Then there is the sheer amazement of what humans were capable of hundreds and thousands of years ago, their craftsmanship, sense of aesthetic and beauty. Coins can be timeless, and their appreciation spiritual. It can also makes you wonder if humans really changed all that much in the last 5000 years or so.
You realize that by the time of the Roman Empire, humankind had experienced exchanging commodities like grain and cattle, COWRIE SHELLS, the most widely "coin" ever circulated, and other types of early “money”, developed coinage, banking systems, knew interest rates, inflation, fixed exchange rates, sales tax, land tax and giro banking.

There are several variables to consider in choosing a coin for collection. One is their eye appeal to you. I recommend you get informed, be patient and enjoy your coins, ancient or new. I personally like ancient coins and generally prefer coins with history to the newly minted ones.

Here are some coin collection info bits:

First coin: By many considered to be the Lydian Lion, ca. 600 BC in Lydia, Asia Minor (current-day Turkey). There is evidence of coinage use in the ancient near east that precedes this coin by many centuries.

Ideas about money originated in Mesopotamia and the Greek colonies in Asia Minor, through which ideas about coinage and much else spread

Numismatics: is the study of coins and currency.
Word origin:
French numismatique, from Late Latin numisma, numismat-, coin, variant of Latin nomisma, from Greek, custom, current coin, from nomizein, to have in use, from nomos, custom. See nem- in Indo-European Roots.

Grading coins: coins get graded according to their technical wear, spots, stains, scratches and dings, eye appeal, provenance, rarity.
There are a number of professional grading services available. Here is a valuable link to coin grading: http://rg.ancients.info/guide/slabs.html

The same coin may get graded differently by the same grading service, that is, be aware that there will be some non objectivity, hence variation, in coin grading.

Know your source when buying a coin, fraud on ebay is common. Don't buy rare coins and don't buy into the “gold coin find”. For beginners and with a bit of patience and looking, you can find a nice ancient coin for $10-20.

Exposure to the air oxidizes metal, changing its color. This does not reduce the value of your coins. It is advisable though, if you are storing your coins for a long period of time, you store them in a controlled environment.
A relatively constant, moderate temperature and low humidity are preferable for long term storage of numismatic collectibles. Placing packets of silica gel in coin storage areas helps control atmospheric moisture.
The more a coin gets handled, the more it degrades.
"Slabs" are sonically sealed hard plastic holders for individual coins. They are best for more valuable coins.
Mylar-line cardboard, often called "2x2s" but also available in other sizes, is a good choice for safekeeping a more average coin.

For interesting US coin trivia, check out:
http://coins.about.com/od/novicecollectors/a/cointrivia1.htm

Guide
Greek and Roman Coins
Grading ancient coins