Kubernetes - Akka clustering Deployment

Question

We have a docker image and a corresponding yaml file for the deployment using kubernetes. The application we have built is in scala with akka-http. We have used akka-cluster. We have a particular variable(seed-nodes in our case - akka cluster) in the configuration file which is used in our application code that uses the pod ip. But, we will not get the pod ip unless the deployment is done. How should we go about tackling the issue? Will environment variables help, if yes how?

More specifically, Once the docker images is deployed in the container in a pod, and when the container starts, the pod ip is already assigned. So, can we programmatically or otherwise configure the pod ips in our code or config file before the process starts in the container?

For reference, this is our configuration file :

akka {
  actor {
    provider = "akka.cluster.ClusterActorRefProvider"
  }    
  remote {
    log-remote-lifecycle-events = off
    netty.tcp {
      hostname = "127.0.0.1"
      port = 0
    }
  }
  cluster {
    seed-nodes = [
      "akka.tcp://our-system@127.0.0.1:3000",
      "akka.tcp://our-system@127.0.0.1:3001",
    ],
    metrics {
      enabled = off
    }
  }    
}
service {
  validateTokenService {
     ml.pubkey.path = "<filePath>"
  }
  ml_repository {
    url = <url address>
  }
  replication.factor = 3
  http.service.interface = "0.0.0.0"
  http.service.port = 8080
}

In the above file, instead of having akka.remote.netty.tcp.hostname as "127.0.0.1", we need to have the pod-ip. So, that we can use it in the seed nodes as :

seed-nodes = [
          "akka.tcp://our-system@hostname:3000",
          "akka.tcp://our-system@hostname:3001",
        ],

How can we do so? Thanks in advance.

Answer 1

I have tried something very similar. What you can do is use stateful sets Kubernetes-Stateful sets. Stateful sets have a naming convention and the pods will be named accordingly - you can read more about that in the link. This way, you can hard-code the values for the seed-nodes, since you know how the pods are going to be named. But, they have a drawback, stateful sets don't yet support rolling update(primarily because they are designed that way.) This article has a great explanation step by step : stateful set - akka clustering This article explains everything together - so posting a link instead of copying the steps here. Hope this helps

Answer 2

As of 2018 the ideal way to initialize a Akka Cluster within kubernetes is the akka-management.

It uses the Kubernetes API to fetch a list of all running pods and bootstrap the cluster. There's no need to configure any seed nodes.

It does also provice a readiness check that waits until the pod has joined the cluster. And a alive check.