I have 2 different versions of the same project on my machine. One from the code trunk, and the other from a code branch. These projects use a .pfx key to enable strong naming.
We had the same problem after doing a "refresh" on Windows 10. Here's what worked for us:
It was like the refresh stored the old containers in some hidden area, where they couldn't be deleted (but also interfered with the addition of the same container). Changing to user-based allows a clean slate from the current user's perspective. Note that if you have multiple users on the same machine, they would probably each need to register the certificates.