The most common culprit for me has been Visual Studio trying to run the tests using a different architecture than the library it's testing. Unfortunately there are multiple places where it seems this can go wrong.
In VS 2017, try creating a Run Settings file, e.g. Default.runsettings
in your test project. If your main lib is x64, the contents should be:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<RunConfiguration>
<TargetPlatform>x64</TargetPlatform>
</RunConfiguration>
</RunSettings>
Then choose this file from Test -> Test Settings -> Select Test Settings File.
Then, under Test -> Test Settings, Default Processor Architecture, choose the correct architecture again.
Be sure to Clean and Build the entire solution. You may need to close and reopen the Test Explorer window. Look for any additional errors in the Output -> Test window, for more clues about incorrect architecture types.
FYI additional Test Settings entries can be found here.