New Guy Just Starting

Probably the cheapest long running set you could complete are Jefferson nickels. Most of the coins you could find through roll searching and even the rarest ones (not counting mint errors) shouldn't cost you more than $20 to obtain.

If money isn't as big of an issue then you have lots of options. I'd suggest getting a RedBook if you haven't already. It is the most respected general price guide for US coins. You should look through it to see what coin styles you like and it will give you an idea of how much it would cost to obtain the coins you want.

Another way to collect might be to obtain a diversity of coins. That's what I did when I started out. I found getting a 2-cent piece, large cent, a Mercury dime, and a Standing Liberty quarter more interesting than getting 4 Mercury dimes from 4 different years.

post pics

penney's are cheap to get and lot's of varieties to look for. one with good pictures is 'VERY' helpful cause there are lost of things that look like they have errors on them when , in fact , the only error is the time you spend learning the hard way that can be very frustrating and time consuming. Much luck to you! I found my first doubled die on the seventh day of searching ten dollars worth of Lincoln Cent rolls! And have found much more. you will to. But watch out, It is very addictive, for myself I know.
Oh, one more thing. You mentioned 'Mercury dimes!' look for a big thrill if you have a 1916 D! The mint mark is on the reverse bottom, Just ot the left of the base of the torch.(hope it is a torch or I wont hear the end of it! I'm a cent guy.....