GKE Ingress Resource with NGINX Load Balancer shows strange IP?

问题

I am running a cluster on GKE where the the ingress is configured to use NGINX like so:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-ingress
  annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/ssl-redirect: "false"
      nginx.ingress.kubernetes.io/use-regex: "true"
....

And I installed the NGINX load balancer on the CLI using Helm. The load balancer console only shows NGINX (and not the Google one), which is good, and my application definitely routes according to my ingress manifest. However, my Ingress shown in the console has the property: loadBalancerIP: xx.xxx.xxx.x and I do not recognize it whatsoever. It's definitely not the external IP used by the NGINX load balancer but it is similar (to where it could be a public IP, not internal). It responds to pings as well. This property was added to the ingress yaml by Google Cloud when it went through the pipeline. Is this anything to be concerned about?

回答1:

I was able to reproduce this behaviour.

1 If you will deploy Nginx Ingress on GKE as per Nginx Docs it is working normally. Service and Ingress have the same IP.

kubectl create clusterrolebinding cluster-admin-binding \
  --clusterrole cluster-admin \
  --user $(gcloud config get-value account)
  
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/cloud/deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
...

2 If you will Deploy Nginx Ingress Helm chart without any changes $ helm install ingress stable/nginx-ingress it will work as you described Nginx ingress controller LoadBalancer service will have one ExternalIP and Ingress will have another ExternalIP.

$ kubectl get svc,ing
NAME                                            TYPE           CLUSTER-IP   EXTERNAL-IP    PORT(S)                      AGE
service/hello-v2-svc                            NodePort       10.8.2.119   <none>         8080:32492/TCP               58s
service/ingress-nginx-ingress-controller        LoadBalancer   10.8.5.90    34.72.141.41   80:32280/TCP,443:31670/TCP   108s
service/ingress-nginx-ingress-default-backend   ClusterIP      10.8.5.66    <none>         80/TCP                       108s
service/kubernetes                              ClusterIP      10.8.0.1     <none>         443/TCP                      169m
NAME                            HOSTS   ADDRESS         PORTS   AGE
ingress.extensions/my-ingress   *       34.66.191.241   80      58s

Regarding part if you should worry it depends. This will not charge you as GKE found only 1 LoadBalancer which is Service LoadBalancer. You can check that by:

$ gcloud compute url-maps list
Listed 0 items.
user@cloudshell:~ (project)$ gcloud compute forwarding-rules list
NAME                              REGION       IP_ADDRESS    IP_PROTOCOL  TARGET
a655d3a06b55511ea89df42010a800fe  us-central1  34.72.141.41  TCP          us-central1/targetPools/a655d3a06b55511ea89df42010a800fe

3 If you want your Ingress and Nginx LoadBalancer service have the same ExternalIP, you must set parameter controller.publishService.enabled to true in helm command. This parameter can be found in Nginx Ingress docs.

controller.publishService.enabled if true, the controller will set the endpoint records on the ingress objects to reflect those on the service false

$ helm install ingress stable/nginx-ingress --set controller.publishService.enabled=true

After that you can deploy some YAMLs like:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-v2
spec:
  selector:
    matchLabels:
      app: hello-v2
  replicas: 1
  template:
    metadata:
      labels:
        app: hello-v2
    spec:
      containers:
      - name: hellov2
        image: "gcr.io/google-samples/hello-app:2.0"
        ports:
        - containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
  name: hello-v2-svc
  labels: 
    app: hello-v2
spec:
  type: NodePort 
  selector:
    app: hello-v2
  ports:
  - port: 8080
    targetPort: 8080
    protocol: TCP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata: 
  name: my-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - http:
      paths:
        - path: /hello-v2
          backend:
            serviceName: hello-v2-svc
            servicePort: 8080
            
$ kubectl apply -f hello.yaml
deployment.apps/hello-v2 created
service/hello-v2-svc created
ingress.extensions/my-ingress created

$ kubectl get svc,ing
NAME                                            TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)                      AGE
service/hello-v2-svc                            NodePort       10.8.3.51     <none>          8080:30572/TCP               19m
service/ingress-nginx-ingress-controller        LoadBalancer   10.8.12.137   34.69.123.145   80:32720/TCP,443:31245/TCP   20m
service/ingress-nginx-ingress-default-backend   ClusterIP      10.8.1.65     <none>          80/TCP                       20m
service/kubernetes                              ClusterIP      10.8.0.1      <none>          443/TCP                      163m

NAME                            HOSTS   ADDRESS         PORTS   AGE
ingress.extensions/my-ingress   *       34.69.123.145   80      19m

$ curl 34.69.123.145/hello-v2
Hello, world!
Version: 2.0.0
Hostname: hello-v2-7cf9b75bbf-2cdj5

来源：https://stackoverflow.com/questions/62520313/gke-ingress-resource-with-nginx-load-balancer-shows-strange-ip