kubernetes Ingress、Ingress controller

前言

拥抱开源，无私分享，共享技术，相互学习，共同进步，分享更多有深度的文章，欢迎转发分享

四层负载均衡调度器service回顾

使用四层负载均衡调度器service时，当客户端访问kubernetes集群内部的应用时，数据包走向如下面流程所示

client--->nodeip:port--->service ip:port--->podip:port

客户端-->node节点的ip:端口--->service的ip:端口--->pod的ip:端口

1.Ingress Controller

Ingress Controller是一个七层负载均衡调度器，客户端的请求先到达这个七层负载均衡调度器，由七层负载均衡器在反向代理到后端pod，常见的七层负载均衡器有nginx，traefik等，以我们熟悉的nginx为例，假如请求到达nginx，会通过upstream反向代理到后端pod，但是后端pod的ip地址是一直在变化的，因此在后端pod前需要加一个service，这个service只是起到分组的作用，那么我们upstream只需要填写service地址即可

nginx：需要手动加载配置文件
traefik：定期自动加载配置文件，不需要手动干预，在微服务中几乎都会使用这种调度器

2.Ingress

官方：https://kubernetes.io/docs/concepts/services-networking/ingress/

ingress官网定义：ingress可以把进入到集群内部的请求转发到集群中的一些服务上，从而可以把服务暴漏到集群外部。Ingress 能把集群内Service 配置成外网能够访问的 URL，流量负载均衡，提供基于域名访问的虚拟主机等；ingress是k8s中的资源，当service关联的后端pod ip地址发生变化，就会把这些变化信息保存在ingress中，由ingress注入到七层负载均衡调度器ingress controller中，也就是把信息传入到七层负载均衡调度器的配置文件中，并且重新加载使配置生效，Ingress 可以用来规定 HTTP/S 请求应该被转发到哪个 Service 上，比如根据请求中不同的 Host 和 url 路径让请求落到不同的 Service 上

总结：

Ingress Controller：

Ingress Controller 可以理解为控制器，它通过不断的跟 Kubernetes API 交互，实时获取后端Service、Pod的变化，比如新增、删除等，结合Ingress 定义的规则生成配置，然后动态更新上边的 Nginx 或者trafik负载均衡器，并刷新使配置生效，来达到服务自动发现的作用。

Ingress：

Ingress 则是定义规则，通过它定义某个域名的请求过来之后转发到集群中指定的 Service。它可以通过 Yaml 文件定义，可以给一个或多个 Service 定义一个或多个 Ingress 规则。

3.七层调度器traefik

如果七层负载均衡调度器使用nginx，那么每次nginx配置文件修改之后，都需要重新reload才能生效，很不方便，如果使用traefik或者Envoy，可以每隔一定时间重新加载配置文件，不需要手动重新加载，对于这种微服务架构来说特别方便

4.使用七层负载均衡调度器的步骤

（1）部署ingress controller，我们ingress controller使用的是traefik
（2）创建service，用来分组pod
（3）创建pod应用，可以通过控制器创建pod
（4）创建ingress http，测试通过http访问
（5）创建ingress https，测试通过https访问

5.客户端通过七层调度器访问后端pod的方式

使用七层负载均衡调度器ingress controller时，当客户端访问kubernetes集群内部的应用时，数据包走向如下面流程所示：
client--->Nodeip:port----->IngressController--->service--->pod
kubernetes Ingress、Ingress controller

6.生成证书，在k8s集群的master节点操作

（1）生成私钥
openssl genrsa -out tls.key 2048
（2）生成证书
openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/O=DevOps/CN=tomcat.lucky.com
（3）生成secret
kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key

Secret 对象类型用来保存敏感信息，例如密码、OAuth 令牌和 ssh key。将这些信息放在 secret 中比放在 pod 的定义或者 docker 镜像中来说更加安全和灵活
kubectl get secret 显示如下：
kubernetes Ingress、Ingress controller

7.部署trafik，在k8s集群的master节点操作

（1）生成openssl.cnf文件，这个文件里面把签发证书时需要的参数已经定义好了
mkdir /root/test
echo """

[req]

distinguished_name = req_distinguished_name

prompt = yes

[ req_distinguished_name ]

countryName = Country Name (2 letter code)

countryName_value = CN

stateOrProvinceName = State or Province Name (full name)

stateOrProvinceName_value = Beijing

localityName = Locality Name (eg, city)

localityName_value = Haidian

organizationName = Organization Name (eg, company)

organizationName_value = Channelsoft

organizationalUnitName = Organizational Unit Name (eg, section)

organizationalUnitName_value = R & D Department

commonName = Common Name (eg, your name or your server\'s hostname)

commonName_value = *.multi.io

emailAddress = Email Address

emailAddress_value = xianchao@qq.com

""" > /root/test/openssl.cnf

（2）生成一个证书
openssl req -newkey rsa:4096 -nodes -config /root/test/openssl.cnf -days 3650 -x509 -out /root/test/tls.crt -keyout /root/test/tls.key
（3）创建一个secret对象
kubectl create -n kube-system secret tls ssl --cert /root/test/tls.crt --key /root/test/tls.key
（4）部署traefik
cat traefik.yaml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: traefik-conf
  namespace: kube-system
data:
  traefik.toml: |
    insecureSkipVerify = true
    defaultEntryPoints = ["http","https"]
    [entryPoints]
      [entryPoints.http]
      address = ":80"
      [entryPoints.https]
      address = ":443"
        [entryPoints.https.tls]
          [[entryPoints.https.tls.certificates]]
          CertFile = "/ssl/tls.crt"
          KeyFile = "/ssl/tls.key"
---
kind: DaemonSet
apiVersion: apps/v1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  selector:
      matchLabels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      tolerations:
      - key: node-role.kubernetes.io/master
        effect: NoSchedule
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      volumes:
      - name: ssl
        secret:
          secretName: ssl
      - name: config
        configMap:
          name: traefik-conf
      containers:
      - image: k8s.gcr.io/traefik:1.7.9
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: admin
          containerPort: 8080
        securityContext:
          privileged: true
        args:
        - --configfile=/config/traefik.toml
        - -d
        - --web
        - --kubernetes
        volumeMounts:
        - mountPath: "/ssl"
          name: "ssl"
        - mountPath: "/config"
          name: "config"
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin
    - protocol: TCP
      port: 443
      name: https
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: ingress.multi.io
    http:
      paths:
      - backend:
          serviceName: traefik-web-ui
          servicePort: 80

把traefik镜像traefik_1_7_9.tar.gz上传到k8s各个节点，手动解压：

docker load -i traefik_1_7_9.tar.gz

镜像地址：

链接：https://pan.baidu.com/s/1fTRc0J0iL7DeAH_LNUkTQQ
提取码：xg1z

kubectl apply -f traefik.yaml

（5）验证traefik部署是否成功
kubectl get pods -n kube-system 显示如下，说明部署成功
traefik-ingress-controller-g45th 1/1 Running 0 9m2s
traefik-ingress-controller-lj9md 1/1 Running 0 9m2s

8.部署tomcat

cat tomcat-deploy.yaml

apiVersion: v1
kind: Service
metadata:
  name: tomcat
  namespace: default
spec:
  selector:
    app: tomcat
    release: canary
  ports:
  - name: http
    targetPort: 8080
    port: 80
  - name: ajp
    targetPort: 8009
    port: 8009
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deploy
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat
      release: canary
  template:
    metadata:
      labels:
        app: tomcat
        release: canary
    spec:
      containers:
      - name: myapp
        image: tomcat:8.5-jre8-alpine
        ports:
        - name: http
          containerPort: 8080
        - name: ajp
          containerPort: 8009

kubectl apply -f tomcat-deploy.yaml
kubectl get pods 显示如下，说明部署成功
tomcat-deploy-59664bcb6f-5z4nn 1/1 Running 0 20s
tomcat-deploy-59664bcb6f-cgjbn 1/1 Running 0 20s
tomcat-deploy-59664bcb6f-n4tqq 1/1 Running 0 20s

9.部署tomcat http

cat ingress-tomcat.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "traefik"
spec:
  rules:
  - host: tomcat.lucky.com
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat
          servicePort: 80

kubectl apply -f ingress-tomcat.yaml

10.部署tomcat https

cat ingress-tomcat-tls.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-tomcat-tls
  namespace: default
  annotations:
     kubernetes.io/ingress.class: traefik
spec:
  tls:
  - hosts:
    - tomcat.lucky.com
    secretName: tomcat-ingress-secret
  rules:
  - host: tomcat.lucky.com
    http:
      paths:
      - path:
        backend:
          serviceName: tomcat
          servicePort: 80

kubectl apply -f ingress-tomcat-tls.yaml

kubectl get ingress

显示如下：
NAME HOSTS ADDRESS PORTS AGE
ingress-tomcat tomcat.lucky.com 80 103s
ingress-tomcat-tls tomcat.lucky.com 80, 443 8s

11.配置自己电脑的hosts文件

192.168.0.16 tomcat.lucky.com

12.在浏览器访问：

https://tomcat.lucky.com:443

或者http://tomcat.lucky.com

可看到如下界面，说明https配置成功
kubernetes Ingress、Ingress controller