So you're trying to crop around a 12-sided coin? In Photoshop, I would try the polygonal lasso tool to select the shape, then create a mask. By using the mask, you can fix any problems with the initial selection and get it perfect.
If you're still stumped, shoot me an email and I'll do an example to drop into your album. This stuff is quick for me. 


















