With Roslyn being open source, code analysis tools are right at your fingertips, and they're written for performance. (Right now they're in pre-release).
However, I can't speak to the performance cost of loading the assembly.
Install the tools using nuget:
Install-Package Microsoft.CodeAnalysis -Pre
Ask your question:
var isValid = Microsoft.CodeAnalysis.CSharp.SyntaxFacts.IsValidIdentifier("I'mNotValid");
Console.WriteLine(isValid); // False