WCF performance, latency and scalability

Question

I\'m trying to port a simple async TCP server in F# to C# 4. The server receives a connection, reads a single request and streams back a sequence of responses before closing

Answer 1

To answer your second question first, WCF will always have a overhead when compared to raw sockets. But it has a ton of functionality (like security, reliablity, interoperability, multiple transport protocols, tracing etc.) compared to raw sockets, whether the trade-off is acceptable to you is based on your scenario. It looks like you are doing some financial trading application and WCF is possibly not fit for your case (although I am not in finance industry to qualify this with experience).

For your first question, instead of dual http binding try hosting a separate WCF service in the client so that the client can be a service by itself, and use the netTCP binding if possible. Tweak the attributes in serviceThrottling element in service behavior. The defaults were lower prior to .Net 4.

Answer 2

WCF selects very safe values for almost all its defaults. This follows the philosophy of don’t let the novice developer shoot themselves. However if you know the throttles to change and the bindings to use, you can get reasonable performance and scaling.

On my core i5-2400 (quad core, no hyper threading, 3.10 GHz) the solution below will run 1000 clients with a 1000 callbacks each for an average total running time of 20 seconds. That’s 1,000,000 WCF calls in 20 seconds.

Unfortunately I couldn’t get your F# program to run for a direct comparison. If you run my solution on your box, could you please post some F# vs C# WCF performance comparison numbers?

---

Disclaimer: The below is intended to be a proof of concept. Some of these settings don’t make sense for production.

What I did:

• Removed the duplex binding and had the clients create their own service hosts to receive the callbacks. This is essentially what a duplex binding is doing under the hood. (It’s also Pratik’s suggestion)
• Changed the binding to netTcpBinding.
• Changed throttling values:
  • WCF: maxConcurrentCalls, maxConcurrentSessions, maxConcurrentInstances all to 1000
  • TCP binding: maxConnections=1000
  • Threadpool: Min worker threads = 1000, Min IO threads = 2000
• Added IsOneWay to the service operations

Note that in this prototype all services and clients are in the same App Domain and sharing the same thread pool.

What I learned:

• When a client got a “No connection could be made because the target machine actively refused it” exception
  • Possible causes:
    1. WCF limit had been reached
    2. TCP limit had been reached
    3. There was no I/O thread available to handle the call.
  • The solution for #3 was either to:
    1. Increase the min IO thread count -OR-
    2. Have the StockService do its callbacks on a worker thread (this does increase total runtime)
• Adding IsOneWay cut the running time in half (from 40 sec to 20 sec).

Program output running on a core i5-2400. Note the timers are used differently than in the original question (see the code).

All client hosts open.
Service Host opened. Starting timer...
Press ENTER to close the host one you see 'ALL DONE'.
Client #100 completed 1,000 results in 0.0542168 s
Client #200 completed 1,000 results in 0.0794684 s
Client #300 completed 1,000 results in 0.0673078 s
Client #400 completed 1,000 results in 0.0527753 s
Client #500 completed 1,000 results in 0.0581796 s
Client #600 completed 1,000 results in 0.0770291 s
Client #700 completed 1,000 results in 0.0681298 s
Client #800 completed 1,000 results in 0.0649353 s
Client #900 completed 1,000 results in 0.0714947 s
Client #1000 completed 1,000 results in 0.0450857 s
ALL DONE. Total number of clients: 1000 Total runtime: 19323 msec

Code all in one console application file:

using System;
using System.Collections.Generic;
using System.ServiceModel;
using System.Diagnostics;
using System.Threading;
using System.Runtime.Serialization;

namespace StockApp
{
    [DataContract]
    public class Stock
    {
        [DataMember]
        public DateTime FirstDealDate { get; set; }
        [DataMember]
        public DateTime LastDealDate { get; set; }
        [DataMember]
        public DateTime StartDate { get; set; }
        [DataMember]
        public DateTime EndDate { get; set; }
        [DataMember]
        public decimal Open { get; set; }
        [DataMember]
        public decimal High { get; set; }
        [DataMember]
        public decimal Low { get; set; }
        [DataMember]
        public decimal Close { get; set; }
        [DataMember]
        public decimal VolumeWeightedPrice { get; set; }
        [DataMember]
        public decimal TotalQuantity { get; set; }
    }

    [ServiceContract]
    public interface IStock
    {
        [OperationContract(IsOneWay = true)]
        void GetStocks(string address);
    }

    [ServiceContract]
    public interface IPutStock
    {
        [OperationContract(IsOneWay = true)]
        void PutStock(Stock stock);
    } 

    [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
    public class StocksService : IStock
    {
        public void SendStocks(object obj)
        {
            string address = (string)obj;
            ChannelFactory<IPutStock> factory = new ChannelFactory<IPutStock>("CallbackClientEndpoint");
            IPutStock callback = factory.CreateChannel(new EndpointAddress(address));

            Stock st = null; st = new Stock
            {
                FirstDealDate = System.DateTime.Now,
                LastDealDate = System.DateTime.Now,
                StartDate = System.DateTime.Now,
                EndDate = System.DateTime.Now,
                Open = 495,
                High = 495,
                Low = 495,
                Close = 495,
                VolumeWeightedPrice = 495,
                TotalQuantity = 495
            };

            for (int i = 0; i < 1000; ++i)
                callback.PutStock(st);

            //Console.WriteLine("Done calling {0}", address);

            ((ICommunicationObject)callback).Shutdown();
            factory.Shutdown();
        }

        public void GetStocks(string address)
        {
            /// WCF service methods execute on IO threads. 
            /// Passing work off to worker thread improves service responsiveness... with a measurable cost in total runtime.
            System.Threading.ThreadPool.QueueUserWorkItem(new System.Threading.WaitCallback(SendStocks), address);

            // SendStocks(address);
        }
    } 

    [ServiceBehavior(InstanceContextMode = InstanceContextMode.PerSession)]
    public class Callback : IPutStock
    {
        public static int CallbacksCompleted = 0;
        System.Diagnostics.Stopwatch timer = Stopwatch.StartNew();
        int n = 0;

        public void PutStock(Stock st)
        {
            ++n;
            if (n == 1000)
            {
                //Console.WriteLine("1,000 results in " + this.timer.Elapsed.TotalSeconds + "s");

                int compelted = Interlocked.Increment(ref CallbacksCompleted);
                if (compelted % 100 == 0)
                {
                    Console.WriteLine("Client #{0} completed 1,000 results in {1} s", compelted, this.timer.Elapsed.TotalSeconds);

                    if (compelted == Program.CLIENT_COUNT)
                    {
                        Console.WriteLine("ALL DONE. Total number of clients: {0} Total runtime: {1} msec", Program.CLIENT_COUNT, Program.ProgramTimer.ElapsedMilliseconds);
                    }
                }
            }
        }
    }

    class Program
    {
        public const int CLIENT_COUNT = 1000;           // TEST WITH DIFFERENT VALUES

        public static System.Diagnostics.Stopwatch ProgramTimer;

        static void StartCallPool(object uriObj)
        {
            string callbackUri = (string)uriObj;
            ChannelFactory<IStock> factory = new ChannelFactory<IStock>("StockClientEndpoint");
            IStock proxy = factory.CreateChannel();

            proxy.GetStocks(callbackUri);

            ((ICommunicationObject)proxy).Shutdown();
            factory.Shutdown();
        }

        static void Test()
        {
            ThreadPool.SetMinThreads(CLIENT_COUNT, CLIENT_COUNT * 2);

            // Create all the hosts that will recieve call backs.
            List<ServiceHost> callBackHosts = new List<ServiceHost>();
            for (int i = 0; i < CLIENT_COUNT; ++i)
            {
                string port = string.Format("{0}", i).PadLeft(3, '0');
                string baseAddress = "net.tcp://localhost:7" + port + "/";
                ServiceHost callbackHost = new ServiceHost(typeof(Callback), new Uri[] { new Uri( baseAddress)});
                callbackHost.Open();
                callBackHosts.Add(callbackHost);            
            }
            Console.WriteLine("All client hosts open.");

            ServiceHost stockHost = new ServiceHost(typeof(StocksService));
            stockHost.Open();

            Console.WriteLine("Service Host opened. Starting timer...");
            ProgramTimer = Stopwatch.StartNew();

            foreach (var callbackHost in callBackHosts)
            {
                ThreadPool.QueueUserWorkItem(new WaitCallback(StartCallPool), callbackHost.BaseAddresses[0].AbsoluteUri);
            }

            Console.WriteLine("Press ENTER to close the host once you see 'ALL DONE'.");
            Console.ReadLine();

            foreach (var h in callBackHosts)
                h.Shutdown();
            stockHost.Shutdown(); 
        }

        static void Main(string[] args)
        {
            Test();
        }
    }

    public static class Extensions
    {
        static public void Shutdown(this ICommunicationObject obj)
        {
            try
            {
                obj.Close();
            }
            catch (Exception ex)
            {
                Console.WriteLine("Shutdown exception: {0}", ex.Message);
                obj.Abort();
            }
        }
    }
}

app.config:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.serviceModel>
    <services>
      <service name="StockApp.StocksService">
        <host>
          <baseAddresses>
            <add baseAddress="net.tcp://localhost:8123/StockApp/"/>
          </baseAddresses>
        </host>
        <endpoint address="" binding="netTcpBinding" bindingConfiguration="tcpConfig" contract="StockApp.IStock">
          <identity>
            <dns value="localhost"/>
          </identity>
        </endpoint>
      </service>

      <service name="StockApp.Callback">
        <host>
          <baseAddresses>
            <!-- Base address defined at runtime. -->
          </baseAddresses>
        </host>
        <endpoint address="" binding="netTcpBinding" bindingConfiguration="tcpConfig" contract="StockApp.IPutStock">
          <identity>
            <dns value="localhost"/>
          </identity>
        </endpoint>
      </service>
    </services>

    <client>
      <endpoint name="StockClientEndpoint"
                address="net.tcp://localhost:8123/StockApp/"
                                binding="netTcpBinding"
                bindingConfiguration="tcpConfig"
                                contract="StockApp.IStock" >
      </endpoint>

      <!-- CallbackClientEndpoint address defined at runtime. -->
      <endpoint name="CallbackClientEndpoint"
                binding="netTcpBinding"
                bindingConfiguration="tcpConfig"
                contract="StockApp.IPutStock" >
      </endpoint>
    </client>

    <behaviors>
      <serviceBehaviors>
        <behavior>
          <!--<serviceMetadata httpGetEnabled="true"/>-->
          <serviceDebug includeExceptionDetailInFaults="true"/>
          <serviceThrottling maxConcurrentCalls="1000" maxConcurrentSessions="1000" maxConcurrentInstances="1000" />
        </behavior>
      </serviceBehaviors>
    </behaviors>

    <bindings>
      <netTcpBinding>
        <binding name="tcpConfig" listenBacklog="100" maxConnections="1000">
          <security mode="None"/>
          <reliableSession enabled="false" />
        </binding>
      </netTcpBinding>
    </bindings>
  </system.serviceModel>
</configuration>

---

Update: I just tried the above solution with a netNamedPipeBinding:

  <netNamedPipeBinding >
    <binding name="pipeConfig" maxConnections="1000" >
      <security mode="None"/>
    </binding>
  </netNamedPipeBinding>

It actually got 3 seconds slower (from 20 to 23 seconds). Since this particular example is all inter-process, I'm not sure why. If anyone has some insights, please comment.

Answer 3

I would say it depends on your goals. If you want to push your hardware as far as possible then it is certainly possible to get 10,000+ connected clients easily, the secret is minimising time spent in the garbage collector and using sockets efficiently.

I have a few posts on Sockets in F# here: http://moiraesoftware.com

Im doing some ongoing work with a library called Fracture-IO here: https://github.com/fractureio/fracture

You might want to check those out for ideas...