No real easy answer to that question. Many times the answer is neither . Unless you have HIGH valued extreme rarities , auction houses can be the absolute wrong place to put your coins. Many dealers myself included Buy a good majority of coins FROM auctions . In fact before the pandemic hit most material IN auction went TO dealers . You can try selling them yourself online , many people do with varying degrees of success from an occasional seller to some like myself who make a living at it . Where to sell is really dependent on the kind of material and the demand there is for it , and how quickly or not you need the funds . One other issue sellers have to consider in addition to associated costs of selling a collection or accumulation of coins , the implication of taxation . This is a major issue if a high value collection is disposed of . That in itself is or can be a HUGE issue .
I don't have any particularly expensive and valuable coins. At this stage, I am selling the "junk" that I collected at the beginning of my journey. Now I sell it all on ebay (for me, this is the easiest way and the fastest way. There are a lot of potential buyers there). I also sell coins there that I no longer need for one reason or another. It makes no sense for me to contact auction houses or local shops (I generally look at local shops with suspicion). I'm sorry, I'm not a rich collector and I don't have expensive coins.