How to set static internal IP to the GKE internal Ingress

问题

I want to create a Internal Ingress for my GKE workloads. I want to know what is the annotation that I can use so that I set a static INTERNAL IP address/name in my ingress.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-https
  namespace: istio-system
  annotations:
    kubernetes.io/ingress.allow-http: "false"
    kubernetes.io/ingress.class: "gce-internal"
    ingress.gcp.kubernetes.io/pre-shared-cert: my-cert
    helm.sh/chart: {{ include "devtools.chart" . }}
    app.kubernetes.io/instance: {{ .Release.Name }}
    app.kubernetes.io/managed-by: {{ .Release.Service }}
spec:
  backend:
    serviceName: istio-ingressgateway-backend
    servicePort: 443

I understand that It will create a Ingress with Internal IP , BUt I want to set a static IP that I have already created in a region/subnet. Is it possible to do so, If yes is there any annotation for the same

回答1:

TL;DR

Currently there is no possibility to configure default Ingress resource with internal static IP in GKE.

There is a workaround for it which entails using the nginx-ingress controller with internal LoadBalancer service.

Please take a look on official documentation:

• Cloud.google.com: Kuberentes Engine: Internal Load Balancing - documentation used for workaround
• Kubernetes.github.io: Ingress-nginx: Deploy - documentation used for workaround
• Cloud.google.com: Kubernetes Engine: Internal Load Balance Ingress

Below I included an example of the workaround with explanation of taken steps.

---

Explanation:

• It's possible to create an internal LoadBalancer with static IP
• Nginx-ingress is using LoadBalancer type of service as an entrypoint
• You can create an nginx-ingress with internal LoadBalancer as told in above bullet points

Steps:

• Download and modify nginx-ingress definition
• Run and check if nginx-ingress-controller service has desired static IP address
• Deploy example app and test

---

Download and modify nginx-ingress definition

By default nginx-ingress definition from official site will have configured service of type LoadBalancer as an entrypoint. By default it will get an external IP address. You can modify/edit service definition to get an internal one.

Please download this YAML and edit the part responsible for service definition below:

A tip!

nginx-ingress is also available to deploy with Helm!.

# Source: ingress-nginx/templates/controller-service.yaml
apiVersion: v1
kind: Service   
metadata:
  annotations: # ADD THIS LINE 
    cloud.google.com/load-balancer-type: "Internal" # ADD THIS LINE
  labels:
    helm.sh/chart: ingress-nginx-2.4.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.33.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  type: LoadBalancer
  loadBalancerIP: 10.1.2.99 # ADD THIS LINE 
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
    - name: https
      port: 443
      protocol: TCP
      targetPort: https
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/component: controller 

Please take a specific look on part in metadata section:

  annotations: # ADD THIS LINE 
    cloud.google.com/load-balancer-type: "Internal" # ADD THIS LINE

as this part will instruct GCP to provision an internal IP address

Also please take a look on:

  loadBalancerIP: 10.156.0.99 # ADD THIS LINE 

as this line will tell GCP to allocate the IP address provided.

Please have in mind that this address should be compatible with the VPC Network that you created your cluster in.

---

Run and check if nginx-ingress-controller service has desired static IP address

After applying whole definition of nginx-ingress you should be able to run the:

• kubectl get svc ingress-nginx-controller -n ingress-nginx

Output of above command:

NAME                       TYPE           CLUSTER-IP   EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller   LoadBalancer   10.60.6.97   10.156.0.99   80:31359/TCP,443:32413/TCP   2m59s

As you can see the EXTERNAL-IP is in fact internal and set to 10.156.0.99.

You should be able to curl this address and get the default-backend of nginx-ingress-controller.

---

Deploy example app and test

This steps are optional and are only showing the process of exposing example app with mentioned nginx-ingress.

YAML definition of Deployment, Service and Ingress:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-app
spec:
  selector:
    matchLabels:
      app: hello
  replicas: 3
  template:
    metadata:
      labels:
        app: hello
    spec:
      containers:
      - name: hello
        image: "gcr.io/google-samples/hello-app:2.0"
---
apiVersion: v1
kind: Service
metadata:
  name: hello-service
  labels:
    app: hello
spec:
  type: NodePort
  selector:
    app: hello
  ports:
  - name: hello-port
    port: 80
    targetPort: 8080
    protocol: TCP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hello-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host:
    http:
      paths:
      - path: /
        backend:
          serviceName: hello-service
          servicePort: hello-port

After applying this resources you should be able to:

• $ curl 10.156.0.99

and be greeted with:

Hello, world!
Version: 2.0.0
Hostname: hello-app-7f46745f74-27gzh

来源：https://stackoverflow.com/questions/62518176/how-to-set-static-internal-ip-to-the-gke-internal-ingress