Depends on what kinds of coins you're looking for. Finding a place where coins are "cheap" can be tricky, because everyone knows that coins are valuable.
Circulation - obviously, you aren't going to get very much that's too exciting or too exotic, but what you do find will be cheap - face value! If you don't handle too much coinage yourself, it never hurts to ask others for help. I know one guy who seems to have every member of his family, every resident of his housing estate and every checkout attendant in his suburb keeping an eye out for unusual coins for him.
Banks - also a great way to get interesting stuff for face value. If you hadn't noticed yet, there's an entire subculture here on the forum of people who go to banks, buy up lots of rolls of coins, open them, fish out anything odd or interesting, roll up the unwanted stuff and send it back to the bank in exchange for fresh rolls.
Another thing to ask about at banks is what they do with the "weird stuff" that's deposited there - foreign coins and notes, NCLT like bullion coins, that sort of thing. Often, bank workers are happy when someone volunteers to take that sort of thing off their hands.
Charities and Churches - people seem to put all sorts of odd things into donation and collection boxes, and often the people who count the money have no idea what to do with the stuff. Try volunteering to help out at the charity, in exchange for the opportunity to keep (or purchase from them for fair value) any weird stuff donated to them. You'll be helping yourself build a collection, and helping the charity to get fair value for the objects donated to them.
Garage Sales, Flea Markets, Antique shops, etc - Very hit-and-miss, but it works for some collectors. Many people have old/foreign coins they want to sell, and garage sales and flea markets are the place many people try to get rid of them. As I said at the start, some people can tend to think their coins are priceless treasures and ask for far more than the coins are actually worth, but other people are quite clueless and are happy to accept only a fraction of what a coin is truly worth. I know another guy about ten years ago who picked up a one ounce palladium coin at a local pawnbrokers for $10 - apparently, they didn't know what "palladium" was!
EBay and other online sales venues - the "virtual flea market" has many of the upsides (and downsides) of a traditional flea market, with the additional problem of not actually seeing in hand the things you're buying, or meeting the people selling them face-to-face. Finding "bargains" on
ebay is harder than it used to be, but still possible.
In both the real and virtual flea markets, knowledge is what will gain you the true bargains. Know your coins, and the bargains will jump out at you.
Coin dealers - find a couple of good, reliable, honest dealers nearby, and visit them regularly. A dealer who knows he has a good, reliable customer will be far more willing to seek out coins on your behalf, and offer you bargains.
Don't say "infinitely" when you mean "very"; otherwise, you'll have no word left when you want to talk about something really infinite. - C. S. Lewis