I have a VS solution and as part of a TeamCity Build, we restore packages from both a private NuGet feed (myget) and the public feed (nuget.org). Most packages restore fine, but
I ran into this problem with one of our build slaves leveraging TFS and Visual Studio.
The way I fixed it was, I opened the solution that wasn't compiling in Visual Studio, right-clicked on the SLN and selected "Enable Restore NuGet Packages"
That prompts a dialog box that you have to accept. After you've done that, you might be good to go. Right-click on the SLN again and run "Restore NuGet Packages", and if that operation succeeds, you're golden.
There's probably some setting you can adjust programmatically when setting up your slave environments, but that's one direct way to fix this kind of problem.