kubernetes unhealthy ingress backend

Question

I followed the load balancer tutorial: https://cloud.google.com/container-engine/docs/tutorials/http-balancer which is working fine when I use the Nginx image, when I try and u

Answer 1

I was having the same issue. Followed Tex's tip but continued to see that message. It turns out I had to wait a few minutes before ingress to validate the service health. If someone is going to the same and done all the steps like readinessProbe and linvenessProbe, just ensure your ingress is pointing to a service that is either a NodePort, and wait a few minutes until the yellow warning icon turns into a green one. Also, check the log on StackDriver to get a better idea of what's going on.

Answer 2

You need to add a readinessProbe (just copy your livenessProbe).

It's explained in the GCE L7 Ingress Docs.

Health checks

Currently, all service backends must satisfy either of the following requirements to pass the HTTP health checks sent to it from the GCE loadbalancer: 1. Respond with a 200 on '/'. The content does not matter. 2. Expose an arbitrary url as a readiness probe on the pods backing the Service.

Also make sure that the readinessProbe is pointing to the same port that you expose to the Ingress. In your case that's fine since you have only one port, if you add another one you may run into trouble.

Answer 3

I thought it's worth noting that this is a quite important limitation in the documentation:

Changes to a Pod's readinessProbe do not affect the Ingress after it is created.

After adding my readinessProbe I basically deleted my ingress (kubectl delete ingress <name>) and then applied my yaml file again to re-create it and shortly after everything was working again.

Answer 4

I was also having exactly the same issue, after updating my ingress readinessProbe.

I can see Ingress status labeled Some backend services are in UNKNOWN state status in yellow. I waited for more than 30 min, yet the changes were not reflected.

After more than 24 hours the changes reflected and status turned green. I didn't get any official documentation for this but seems like a bug in GCP Ingress resource.