The metal value of a coin has never been a determinant in the deciding to buy, although with gold, it has been a contributing factor. Even then, gold value has not been a contributing factor in the decision a non modern gold coin.
Why all of this statement?
I guess that I have never been a bullion trader, and only rarely have I traded coins for others. I have almost always bought a coin that is interesting to me, and has had good numismatic value.
The result is that over decades of collecting, the value of my total collection is worth many times what I have paid for it. And that is come about despite the fact that numismatic investment has never really been a motivation, except perhaps at the time of the original purchase.
I haven't really cared that much how the value of my collection has varied. I just feel fortunate that I have some of the coins I had earlier purchased, because I cannot afford them now.
For me, having that attitude when attending coin shows has served me well.