The rarity number assigned by Leroy Van Allen is an estimate of the rarity at the time the coin is discovered. He bases this on a few things, including how close to retirement the die seems to be and how likely a new variety submitted to him should have been discovered already. Based on this, you get coins like the 21-D
VAM 1I, 82-CC
VAM 3D, and 82-O
VAM 38B that all look like they should have been retired soon after making their respective coins, but stayed in service long enough to produce plenty of coins. These are all rated R7, but are all much more common. Coins like the ultra-rare 8TFs were given high rarity scores because that series had been so well scrutinized, and it would seem they should have been discovered. Also, terminal die cracks and breaks paired with sharp, prooflike surfaces means the dies did not last long. Finally, something like the 98-O
VAM 20 was such a jaw-dropper when discovered that even not being in a popular series, had that coin not been rare, someone would have spotted it long before it was discovered.
The best way to see rarity is on the SSDC registry, comparing with coins that would likely be in the same collection (compare the 21-D
VAM 1I with other similar 21-D die breaks, for example).
As for the premium for a rare variety, that has to do with demand. There are some rare varieties that nobody cares about, so they carry no premium. It could also be a coin is sort of scarce, but there are enough to go around, so the premium is quite small. Best way to get the premium to go up is to introduce more people to
VAM collecting. If anyone wants to chat about it at the
ANA Summer Seminar, I'll be there during week 1 as an instructor for the
Morgan dollar course.