I'll share my thoughts, which differ from many of those above. One of the many things about our wonderful hobby is often there's no right or wrong. YOU collect how YOU choose to collect, and that's ok.
1. Instead of a complete 24 coin date and mint set, many collectors choose to collect
Peace dollars as a DATE Set, which then is a 10 coin set. This way YOU choose which Mint to get for each year. You can choose a tougher Mint, or if you choose a more common Mint, you can then afford a higher grade, not only because it's a more common mint, but you only have to budget for ten coins instead of 24. These higher grade coins, perhaps MS63 or 64, should then have less wear AND nicer eye appeal.
2. While there's merit to @John1's point about having a "matched" set, and many collectors agree, many collectors, including myself, disagree. It's a personal preference. While the expensive Key dates should only be bought in the grades you can afford, my preference is to not limit yourself to low grades for the common coins, which can easily be bought in MS63 and 64 at relatively low prices. Coins in those grades will have fewer marks if you're patient, and bright cartwheel or lovely satin luster. Those relatively inexpensive coins will have much nicer eye appeal for just a few dollars more than the same date in XF. Having coins with nicer eye appeal in your opinion may give you more pleasure from your set.
3. The final area I disagree with is you taking the time to become an expert in identifying the authenticity of each of the key dates. Instead, I suggest only buying coins already graded by NGC or PCGS. Not only will they guarantee the authenticity, but when the time comes to sell, you won't have a dealer saying your key date XF 45 is really a VF 35. Additionally, the holder provides some environmental protection, as well as physical protection against dropping and/or sloppy handling. Sometimes I have the dropsies, so I'm glad the coin is in that hard holder! A final advantage is many of the coins graded by those firms will have a professional "True View" photo of the obverse and reverse taken while they had the coin raw.
4. I suggest buying these coins from GC (Great Collections). As a generalization the high bid is usually fair, and you'll also get two professional slab shots of each coin (one obverse, the other reverse).
VERY RECENT sales of the common dates graded MS64 and MS64+ have ranged from $50 - $95, INCLUDING the Buyers Premium. Coins totaling less than $500 in a single auction are shipped for only $5 plus 50 cents per coin.
How can you not buy these lovely, highly lustrous coins, professionally graded and authenticated, with professional photos, at these prices, for $50 - $95, graded MS64 or 64+?
Happy Hunting! Have FUN!
Not that it matters, but for ME, I decided my
Peace dollar collection would be a 10 Coin DATE Set, as that allowed me to buy coins in higher grades than I would have been able to buy had I needed to budget for a 24 coin set.
Steve