I read the MSDN article on the topic. To quote:
Because a service must be run from within the context of the Services Control Manager rather than
You could add some logging around the interop calls to find out which one fails.
Also services by default aren't associated with a desktop; if you open the services.msc control panel applet, get the properties of your service, go to the "Log On" tab, you could check "Allow service to interact with desktop". This could fix the problem for you in some cases.
read the 2 articles mentioned here:
http://geekswithblogs.net/BlackRabbitCoder/archive/2011/03/01/c-toolbox-debug-able-self-installable-windows-service-template-redux.aspx
You can use WinDbg/NTSD (another debugger from the "Debugging tools for windows" package) to start a debugger together with your service.
To do this open "gflags" (also available in the above mentioned package) to the "Image file" tab and set the path to debugger executable for your image file (service);
If your service is marked as interactive (only possible if it runs under the SYSTEM account) you can directly start WinDbg, just set the debugger to something like "PATH_TO_WINDBG\windbg.exe -g -G" (the -g / -G are needed so that the debugger doesn't break execution on application start or end - the default behaviour). Now when starting your service the windbg window should pop-up and will catch any unhandled exception.
If your service is not interactive you can start the NTSD debugger (a command line debugger) in remote mode and connect to it from WinDbg (that can even be running in another PC). To do this set the debugger in gflags to something like "PATH_TO_NTSD\ntsd -remote tcp:port=6666,server=localhost". Then connect to the remote debugger by starting windbg with something like "windbg -remote tcp:port=6666,server=localhost" and you should have complete control over the other debugging session.
As for finding the source of the exception itself a windbg tutorial is over the topic here but as a start try to execute the "!analyze -v" command after the exception was caught - with some luck this is all information you'll need..
Note: maybe this is overkill for your case but with this approach you can even debug services during system start-up (I had once a timing problem with a service had an issue only when starting the first time with the system)
Add lots of verbose logging in your OnStart. It's painful and old school, but it works.
I usually just manually set a breakpoint, then point it to the currently open project in c#. The code to set a breakpoint is:
System.Diagnostics.Debugger.Break();
That should get you started, then you can just step through your code and see what's really happening.
I would assume the reason could be causing because of heavy use of interops. So you need to tackle this problem differently. I would suggest create a windows or console app with same logic of you service and make sure that it works first without any issues, and then you may want to go with creation of the Win service.