Tape Residue

Use PURE acetone,Wal-Mart has it for $2.50 a bottle over by the nail polish.
John1

As already said use Acetone. However do not use the stuff from the fingernail polish remover. Although many say pure Acetone, many still contain perfuming agents for women. Some even contain miscellaneous other items that are in such small quantities, government regulations allow them to ignor adding those to the lables.
A can from the paint department usually is about $4 to $5 and is just Acetone. Nothing needs to be added for paint removal.
The main thing is to use the Search tab on how to use it properly.
Also, as noted, even if you get it all off, probably will be a different colored area where that tape used to be.

Update: Acetone worked to get the residue off (had to leave the coins in it overnight though - it was that bad!). You were right about the differential toning - stripes are definitely present where the tape had been.

Next question: What type of dip to use? I see Wal-mart has jewelry dipping solutions for sale, including one specifically marked for silver. Would this work? When my LCS has dipped stuff for me in the past, it looks like the same blue solution they sell for dipping jewelry (as they also sell and repair jewelry). Thoughts?

I ended up picking up a jar of Connoisseurs Silver dip. It worked great! Both halves came out beautiful, and both have great cartwheeling. There's still some barely noticeable residue left on the 1968 which I don't think is going to come off with anthing short of steel wool. But you'd have to really be looking hard for it in the right place to see it.

For fun, I also tried it out on a junk Mercury which was nearly black - came out nicely! Also on a Wheat cent which came out nearly pink, and then this morning was bright orange.