The GPL is a license for the distribution of software owned by the original copyright holder, as long as you don't distribute GPL code yourself you can ignore it.
If you distribute GPL code together with GPL-incompatible code, you are violating the license and thus, you are not getting a right to copy which means you are committing a copyright violation over the original GPL code. Your code is still copyright you and available under whatever license you choose, unless it is a derivative work.
The GPL asserts that anything that links with GPL code is a derivative work, but that is dubious to say the least especially if the interfaces to that code are public.
However, if their assertion was supported in court, and for example under Japanese Copyright Law, the original authors would get copyright over all of your code except the moral right to modification meaning they would be free to distribute free copies of your code whatever license you used for it even if you never distributed GPL code.
Note that even if your code is not derivative despite of linking, you would still be violating the copyright of GPL code on distribution. You could distribute it to users of the GPL program for them to do the linking, though.
Even blurrier is what happens to code that was originally derived from other code but no longer containing any of the original code. Most people seem to think that deleting the code is enough, but is it? No one knows for sure.
