Windows service locks up on assembly load

Question

I have a Windows Service written in C#. It includes a standalone console mode as well, for debugging purposes. It works fine on almost every computer it's been run on, but we ran into a situation where this service locks up when you try to start it, and then it gets killed because of timeout. But when running it in console mode on the same machine, it starts up fine.

It's a pain to debug because I don't actually have access to the machine this is happening on, I have to go through a human proxy. But after a bunch of trial and error debugging, I finally narrowed the cause down to assembly load. When it hits the first reference to any data type in a specific dll, it stops right there, according to the log file. It's not even giving an exception, it just locks up.

[Edit] After further examination, it appears it's not locking up permanently, it just takes about 40 seconds to actually finish loading the library, which is long enough for Windows services to decide to kill the process.

Any clue how to debug this kind of situation?

Here's about the simplest solution that I can reproduce it with. "Before" shows up, but "During" and "After" do not.

private static void LoadAssembly()
{
    Log("During");
    MyNameSpace.MyClass x = new MyNameSpace.MyClass();        
}

static void Main(string[] args)
{
    try
    {
        // Leaving out code to handle command line parameters
        // ...
        //

        Log("Before");
        LoadAssembly();
        Log("After");
        if (Environment.UserInteractive)
        {
            Log("Starting in console mode");
            ConnectionManager.Listen();
        }
        else
        {
            Log("Starting in service mode");
            ServiceBase.Run(new RunAsService());
        }

    }
    catch (Exception ex)
    {
        Log(ex.ToString());
    }
}

Answer 1

The assembly was taking 45 seconds to connect, it tries to connect to the internet to verify something about the assembly before loading it, but on that particular machine it's blocked by the firewall. After 30 seconds of trying to start, Windows service manager gives up and kills the process.

Moving the loading of the assembly until after the service has been started allowed it to start up properly (although with a 45 second delay before it starts responding).

It looks like it was related to generating publisher evidence. From here http://msdn.microsoft.com/en-us/library/bb629393.aspx:

We recommend that services use the element to improve startup performance. Using this element can also help avoid delays that can cause a time-out and the cancellation of the service startup.

Putting the following in the app.config file got rid of the delay:

<configuration>
    <runtime>
        <generatePublisherEvidence enabled="false"/>
    </runtime>
</configuration>

Answer 2

Are you doing things on module load while holding the loader lock?

See this article for debugging: http://msdn.microsoft.com/en-us/library/ms172219.aspx