It does depend on the country. Australian 50 cent coins, for example, are struck on dodecagonal-shaped blanks. While any degree of rotation is possible, the most commonly encountered ones are the exact multiples of 30 degrees, so the sides line up again perfectly.
Example with 30 degree rotation. In your hypothetical square coin, only multiples of 90 degree rotations are "stable". Rotations at any other angle are likely to result in a mangled coin that gets stuck in the dies, so they're much harder to come by.
What you can also get when you use blanks that are pre-shaped is a misalignment or upset of the blank itself in the dies. This can look much like what you might expect a slightly rotated die to look like, except of course the dies aren't rotated and you can see damage on both sides of the coin.
Example.
Don't say "infinitely" when you mean "very"; otherwise, you'll have no word left when you want to talk about something really infinite. - C. S. Lewis