As a beginner in the field, I'd recommend that you stay away from eBay - there's way too many fakes and raw deals there.
For the most part, I buy my ancients and mediaevals off of local dealers that stock them; I don't buy too much from the Internet at all. However, you might not have any local dealers that stock them, so the Internet will be your only recourse.FORVM Ancient Coins
is one of the bigger ancients specialists online, and the only people I've heard talk them down are the people they accuse of selling fakes on eBay; they maintain a "blacklist" of eBay sellers of fakes. They sell all kinds of ancients, from cheap common stuff and uncleaned coins to the scarce and beautiful.Civitas Galleries
is another US-based dealer I've heard only good things about. They stock ancient, mediaeval and modern coins. I've bought a couple of coins off them.
As for what to get: the cheapest ancient silver coins would be Roman denarii, from about 90 to 200 AD. Typical examples from this period will cost as little as $50, maybe even less.
The most readily available mediaeval silver coins here in the English-speaking world will probably be English hammered silver pennies, though other European countries are also fairly easy to find. They can be obtained for $20 or so, though if you want them to be recognizable you might have to pay twice that; mediaeval coins are typically very thin, and more easily damaged than the thick, chunky ancient coins are.
Don't say "infinitely" when you mean "very"; otherwise, you'll have no word left when you want to talk about something really infinite. - C. S. Lewis