When in doubt, always prefer an implementation that's included in the .NET framework. You can have high expectations that such an implementation is tested by hundreds of thousands of programmers, has been carefully reviewed for security and usability and will be maintained for years to come.
The mutex approach is an easy one to get going. It however suffers from a pretty severe security problem. A denial of service attack is very simple to get going, you cannot keep the name of your mutex a secret and anybody can trivially create a mutex with the same name and prevent your program from ever starting up.
The process name approach is deeply flawed for the same reason. There is no guarantee that a process name is unique. Not just easy to exploit but easily triggered by accident.
WindowsFormsApplicationBase has an image problem in the eyes of C# programmers. They choke at the namespace name and assume that their program will somehow be infected with vb-isms. That's nonsense, it is just a plain .NET class that's useable in any language.
