How do you set up two microservices that both use https port?

Question

I have one microservice that is a Asp.Net Core stateless service. The service runs fine and i can access at https://myazureresource.eastus.cloudapp.azure.com/controller/method.

My ServiceManifest.xml is:

<Resources>
  <Endpoints>
    <Endpoint Protocol="https" Name="EndpointHttps" Type="Input" Port="443" />      
  </Endpoints>
</Resources>

And MyService.cs is:

protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
    return new ServiceInstanceListener[]
    {
                new ServiceInstanceListener(serviceContext =>
                    new KestrelCommunicationListener(serviceContext, "EndpointHttps", (url, listener) =>
                    {
                        var transportCertificate = GetCertificate(ConfigurationManager.Instance["mycert"]);

                        return new WebHostBuilder()
                                    .UseKestrel(x =>
                                    {
                                        int port = serviceContext.CodePackageActivationContext.GetEndpoint("EndpointHttps").Port;
                                        x.Listen(IPAddress.IPv6Any, port, listenOptions =>
                                        {
                                            listenOptions.UseHttps(transportCertificate);
                                            listenOptions.NoDelay = true;
                                        });
                                    })
                                    .ConfigureServices(
                                        services => services
                                            .AddSingleton<StatelessServiceContext>(serviceContext))
                                    .UseContentRoot(Directory.GetCurrentDirectory())
                                    .UseStartup<Startup>()
                                    .UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
                                    .UseUrls(url)
                                    .UseSerilog(Logger.Serilog)
                                    .Build();
                    }))
    };
}

Everything works fine however now I am not trying to create another microservice. I get an error that it can't bind to 443 obviously because it is already being used. Somehow I have to tell it to use the name of my service in the url.

I've tried many things including appending to the url:

.UserUrls(url + "/myservice")

but i get an exception that it can't bind because it is overriding and using UseKestrel endpoints. I then tried PreferHostingUrls(true) but then got an error that you have to use UsePathBase.

I then try:

.UsePathBase("myservice")

and I then get an error that it does not have a valid certificate for https. But it works with one microservice.

1. Does anyone know what you have to do to get two microservices working at the same time?

2. On a separate note (and just for curiosity) I always wondered why you would ever use dynamic ports. If the service is selecting the port how would you ever know what url to give to your clients?

Thanks for any help.

Answer 1

I'm answering this in case someone else has the same question. Not sure this is the only way but the way i solved it was to use a reverse proxy. You could also use Application Gateway which is more powerful but for my needs i just needed the reverse proxy.

I had to recreate my service fabric (couldn't get it to use a reverse proxy after the fact). Select reverse proxy when creating the service fabric.

Note: if you want https, create a key vault first and add two certificates. on the certificates page select 'Custom' and use different ones for the cluster and reverse proxy. Azure fails deployment if you use the same one.

Also select at least a D1_s2 size machine or Azure will hang indefinitely when deploying and not tell you anything. I have no hair left after this.

when you call the reverse proxy you specify the fabric and service:

https://.eastus2.cloudapp.azure.com:19081/MyFabric/MyService/Controller/route

when talking from one service to another you can use localhost:19081 instead of the url.

in your servicemanifest.xml set the endpoint something like:

<Resources>
  <Endpoints>
    <Endpoint Protocol="http" UriScheme="http" Name="EndpointHttp" Type="Input" />
  </Endpoints>
</Resources>

the proxy knows where to send to. I set up an Application Management API that forwards the request to the proxy. Note this is for a single partition cluster. if you have multiple partitions you need to specify the partition id in the url also.

https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-reverseproxy

Answer 2

To answer your questions:

1. Yes, there is a built-in reverse proxy in Service Fabric that you can use. As you are on Kestrel, using a reverse proxy would be the recommended approach, although not necessary. This would look like:

   

   Kestrel used as an edge server without a reverse proxy server doesn't support sharing the same IP and port among multiple processes. When Kestrel is configured to listen on a port, Kestrel handles all of the traffic for that port regardless of requests' Host headers. A reverse proxy that can share ports has the ability to forward requests to Kestrel on a unique IP and port.

   Here are some references:

   • https://github.com/Azure/service-fabric-issues/issues/1339
   • https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-reverseproxy
   • https://docs.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel?view=aspnetcore-2.2#when-to-use-kestrel-with-a-reverse-proxy

2. In a distributed system, services may move from one machine to another over time. This can happen for various reasons, including resource balancing, upgrades, failovers, or scale-out. This means service endpoint addresses change as the service moves to nodes with different IP addresses, and may open on different ports.

   Service Fabric provides a discovery and resolution service called the Naming Service. The Naming Service maintains a table that maps named service instances to the endpoint addresses they listen on. All named service instances in Service Fabric have unique names represented as URIs, for example, "fabric:/MyApplication/MyService". The name of the service does not change over the lifetime of the service, it's only the endpoint addresses that can change when services move. This is analogous to websites that have constant URLs but where the IP address may change. And similar to DNS on the web, which resolves website URLs to IP addresses, Service Fabric has a registrar that maps service names to their endpoint address. That way, we wouldn't have to worry about service discovery and resolution as it is offloaded to a reverse proxy or a Load balancer.

   Here are some great articles for your reference:

   • https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-connect-and-communicate-with-services
   • https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-reverseproxy (again ;) )

Hope this helps.