Where To Get Some Old Coins

Do your price research on VCOINS.
Also a good site for consideration to buy some better quality ancient coins.
WILDWINDS is also an excellent reference website.
Maintain good connections with like minded and knowledgeable ancient coin collectors here in the CCF.
Calgary Coin in Canada seem to be excellent at what they do, and know their subject well.

Get yourself at least ONE good book on Roman coins.
The best beginner's book is the one volume reprint of
Roman Coins and Their Values by David Sear.
These days, I have over a dozen good reference books on ancient coins.

Learn all you can about fake ancient coins, and maintain a fake ancient coin collection so that you can become very familiar with them. Use the products of dishonest people as a weapon against them. A good reference book on this vexed subject is
Numismatic Forgery, by Charles M. Larson. Lots of good information on how dangerous fakes are made, and how to identify them.

The majority of ancient coins valued at $500 or more in my collection, have come from leading Australian auctions.
I always like to go to the pre auction view days, to assess each coin 'in hand' first, before submitting a written single price bid. I have submitted hundreds of low bids, to be successful in receiving tens of coins, over about a 30 year period.
A tiny number of them have come direct from leading dealers in London, such as Spink's and Baldwin's.

For lower valued ancient coins, say up to $100, ebay can be an excellent way of acquiring interesting Roman coins.

For ancient coins, I'd suggest that newbies stay away from ebay completely, as there are lots of fakes out there. If you must venture out into "the Bay", then at least check out the FORVM Notorious Fake Seller List and make sure your prospective seller isn't listed there.

The coin series known as "Late Roman Bronzes", dating roughly from 300 to 400 AD, are the cheapest and most abundant ancient coins you're likely to come across; it's possible to buy identifiable (though not very nice-looking) examples for just a few dollars each. The artwork's not as pretty and "classical-looking" as it is on Greek and earlier Roman coins, but they're still highly collectable; the use of easily translatable mintmarks also gives added variety and interest to this series.

There's no one series of late mediaeval / early modern European that stands out in my mind as "easy to collect" or "cheap", so it really depends on where your interest lies. Many folks from the English-speaking world like to collect English hammered silver pennies, but if your personal ancestral roots lie elsewhere in Europe, you might find coins from those places to be of more interest to you.

As for the late Byzantine period, there's not a lot to offer. The declining years of the Empire saw very little coinage produced and what they did make was rather crude-looking, so it has the unfortunate combination of appearing both "ugly" and "expensive". The series of Anonymous Bronzes, dating from roughly AD 900-1100, are the closest you'll come to "cheap late Byzantines".

Of course, coinage from the "other side" - the Islamic world during this time period - is fairly cheap and plentiful by comparison, but they're usually considered outside the scope of "European" coinage, even though many Ottoman coins were made in "European" mints.

In France that would be the period after the Fronde, but most those coins are identified by the rulers.
My suggestion would be to get some reference books, a second hand copy will do (if you can get one).
Eg for France http://www.amazon.com/Coinage-Franc...p/090065287X and for England http://www.amazon.com/Coins-England...p/1902040767 (older issues were issued by Seaby rather than by Spink). But beware, while the volume on England is pretty complete for the period you are looking at, France had a significant amount of feudal/local coinage which is only scratched in Mayhew.