I posted a similar question over the summer. The reply I got was
Ashbrook Collectibles on Richmond St.
Universal Coins on Bank St.
Both are beyond walking distance from the Parliament. There is also Alliance Coin and Banknote in Almonte which is about 20 minutes by car outside of Ottawa but worth the trip. And if you're traveling along the 401 to get there, I'm in Brockville on King St. Also be sure to check out the
RCM while there. They have short tours.
Unfortunately I did not get the chance to visit them, but I did take a tour of the mint. It was pretty cool, but the Ottawa mint only does NCLT and I think maple leaf coins (silver, gold, etc).