WCF Service returning “requested service '…' could not be activated” the first time it's accessed from an MVC site

Question

We have a WCF service (with no security) that is being accessed by an MVC3 website.

On the developer machines we have no problems with it but when our TeamCity setup bui

Answer 1

I've just manage to solve similar problem of random 'System.ServiceModel.ServiceActivationException' (thrown exception, then run on second hit.). The reason for the exception in my case was lack of memory on the server. To confirm the memory problem, you can do following:

Application calling web service:

System.ServiceModel.ServiceActivationException: The requested service, 'XXX' could not be activated. See the server's diagnostic trace logs for more information.

Server stack trace: 
   at System.ServiceModel.Channels.HttpChannelUtilities.ValidateRequestReplyResponse(HttpWebRequest request, HttpWebResponse response, HttpChannelFactory`1 factory, WebException responseException, ChannelBinding channelBinding)
   at System.ServiceModel.Channels.HttpChannelFactory`1.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

WCF trace log: nothing logged.

On server hosting wcf service (the same as the calling application btw) in event logs except lot of crap info I've found:

Custom event details:

WebHost failed to process a request.
 Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/58366981
 Exception: System.ServiceModel.ServiceActivationException: The service 'xxx' cannot be activated due to an exception during compilation.  The exception message is: Memory gates checking failed because the free memory (xxx bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.. ---> System.InsufficientMemoryException: Memory gates checking failed because the free memory (xxx bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.

So solution for me was: 1. add more ram 2. release some of ram (temporary solution) 3. update web config of the web service: