You'll always have to crop the pictures manually if you want the final image to be square, like you avatar. No cameras produce square photos. But I think the issue is the coin is too small in the frame, correct?
Take a look at the "<$400" thread, which for your purposes (and with price erosion since it was published) is probably now a "<$300" or maybe even "<$250" thread. For somewhere around that price, you can buy a used DSLR, extensions and such, plus a decent lens.
How are you holding your Powershot steady for photos now?
edited to add: here's the thread I mentioned:
http://goccf.com/t/158182&whichpage=1My post where I outline the components you need to buy is the 5th one down. The list is:
Canon XS camera, body only: $175
EOS-M42 Adapter: $5
M42 focusing helicoid, 17-31mm: $30
M42 extensions, 2 sets 7/14/28mm: $15
M42-M39 adapter: $3
M42-RMS Adapter: $15
Nikon 75mm EL-Nikkor Lens: $40
Nikon M5 or Plan4 objective: $30
Tripod: $30
IKEA Jansjo LED lights, two: $20
Total $363
Today, you can probably get the XS camera for ~$125, and for your purposes you don't need the M5 objective or adapter, so that reduces the total by $95 to $268.