The speculation that One of the Kings mistress's was the model for Britannia is nothing but speculation, Britannia re-emerged on Brit coins in 1672 on Charles II copper halfpennies and farthings, and speculation by Samuel Pepys at the time was because of the likeness of Britannia on these coins, he believed was intended to be Frances Teresa Stewart (or Stuart) , later known as the Duchess of Richmond. Frances was one of Charles's II mistress's at the time,and also later, but there is/was no direct evidence to support the theory.
When Britannia made her re-appearance in 1672 coins. she is shown seated on a rock, facing left, with an olive branch in her right hand, a spear in her left hand, and a shield leaning against the rock. The shield bears a Union flag. The design and attributes of this portrayal of Britannia are almost certainly inspired by the earlier Britannias who appeared on the much earlier Roman coins.