The advise already given is excellent, particularly about keeping track of what you have and not looking at this as an investment.
That said, I would say I disagree with not going out "collecting everything". If you are just starting, this is not the most useful advise. I would say it is worthwhile learning the relative values (what something is normally selling for) before you buy. I have a many "series" that I collect, other items I clump by categories I deem relevant "i.e. ones that have specific designs like birds or castles, etc.) and some that I acquired just because I like the design.
My message is, if you have a diverse collection that doesn't fit any one else's definition of what you should collect, that's fine as long as you like it. I don't regret anything I have in my collection, because I don't take time to regret things. Learn more, move on, and you will have a great collection that fits who you are.
Best of luck in all your endevours!