部署 dashboard
由于会被墙, 所以要加一步拉取镜像
docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1 docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/kubernetes-dashboard-amd64:v1.10.1 k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1 #把镜像copy到其他节点 docker save k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1 -o dashboard.tar scp dashboard.tar 10.0.0.51:/root/ #在其他node节点导入镜像 docker load -i dashboard.tar docker image ls|grep k8s.gcr.io/kubernetes-dashboard-amd64
正式开始安装dashboard
[root@master ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml [root@master ~]# kubectl apply -f kubernetes-dashboard.yaml secret/kubernetes-dashboard-certs created serviceaccount/kubernetes-dashboard created role.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-minimal created deployment.apps/kubernetes-dashboard created service/kubernetes-dashboard created
查看
[root@master ~]# kubectl get pods -n kube-system NAME READY STATUS RESTARTS AGE coredns-5c98db65d4-8mzfz 1/1 Running 0 44d coredns-5c98db65d4-spjx8 1/1 Running 0 44d etcd-master.kubernetes 1/1 Running 0 44d kube-apiserver-master.kubernetes 1/1 Running 0 44d kube-controller-manager-master.kubernetes 1/1 Running 0 44d kube-flannel-ds-amd64-4szk7 1/1 Running 0 44d kube-flannel-ds-amd64-b4ssp 1/1 Running 1 44d kube-flannel-ds-amd64-nmklz 1/1 Running 0 44d kube-flannel-ds-amd64-wjczq 1/1 Running 0 44d kube-proxy-8fqsz 1/1 Running 0 44d kube-proxy-bkrw4 1/1 Running 0 44d kube-proxy-n75g8 1/1 Running 1 44d kube-proxy-rmckk 1/1 Running 0 44d kube-scheduler-master.kubernetes 1/1 Running 0 44d kubernetes-dashboard-7d75c474bb-cqwh6 1/1 Running 0 18s # 已经启动成功
开放访问
dashboard默认采用的是ClusterIP, 用patch修改type为NodePort, 以便可以直接访问(也可以采用ingress)：
[root@master ~]# kubectl patch svc kubernetes-dashboard -p '{"spec":{"type":"NodePort"}}' -n kube-system service/kubernetes-dashboard patched [root@master ~]# kubectl get svc -n kube-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 44d kubernetes-dashboard NodePort 10.106.202.90 <none> 443:32211/TCP 3m16s 通过查看，dashboard的svc已经被映射到32211 这个端口，所以在浏览器直接输入任意节点的ip + 端口即可访问。
注意访问的地址是 https://IP:PORT，注意一定要是https,否则就会显示乱码
配置dashboard用户
Dashboard 登陆有两种认证方式：
- kubeconfig 文件认证
- token 令牌认证
下面对这两种认证都进行测试验证
1. token 令牌认证
创建一个 serviceAccount
在 kube-system 名称空间中创建
[root@master ~]# kubectl create serviceaccount dashboard-admin -n kube-system [root@master ~]# kubectl describe sa/dashboard-admin -n kube-system Name: dashboard-admin Namespace: kube-system Labels: <none> Annotations: <none> Image pull secrets: <none> Mountable secrets: dashboard-admin-token-wrdvz Tokens: dashboard-admin-token-wrdvz Events: <none>
dashboard-admin 绑定 clusterbinding 的 sa
[root@master ~]# kubectl create clusterrolebinding dashboard-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:\dashboard-admin clusterrolebinding.rbac.authorization.k8s.io/dashboard-cluster-admin created [root@master ~]# kubectl describe clusterrolebinding/dashboard-cluster-admin Name: dashboard-cluster-admin Labels: <none> Annotations: <none> Role: Kind: ClusterRole Name: cluster-admin Subjects: Kind Name Namespace ---- ---- --------- ServiceAccount dashboard-admin kube-system
这帮绑定后，serviceaccount 的 dashboard-admin 就有集群的管理员权限了。
获取 serviceaccount 的 dashboard-admin 的secret信息
在创建好serviceaccount 的 dashboard-admin 后，会在 kube-system名称空间下的secret 中，自动创建一个 以 dashboard-admin 开头的 secret，携带者 token 字样
[root@master ~]# kubectl get secret -n kube-system NAME TYPE DATA AGE attachdetach-controller-token-7nxx6 kubernetes.io/service-account-token 3 44d bootstrap-signer-token-c44r9 kubernetes.io/service-account-token 3 44d bootstrap-token-cgw8vp bootstrap.kubernetes.io/token 7 44d certificate-controller-token-6682d kubernetes.io/service-account-token 3 44d clusterrole-aggregation-controller-token-g7l5s kubernetes.io/service-account-token 3 44d coredns-token-gsrxq kubernetes.io/service-account-token 3 44d cronjob-controller-token-qch7l kubernetes.io/service-account-token 3 44d daemon-set-controller-token-gmp2z kubernetes.io/service-account-token 3 44d # 下面这个就是自动生成的 dashboard-admin-token-kzf2m kubernetes.io/service-account-token 3 6m50s default-token-ll59x kubernetes.io/service-account-token 3 44d ... ... ... ...
从上已经查看到了，下面需要查看一下详细信息即可获取到token的值
[root@master ~]# kubectl describe secret/dashboard-admin-token-kzf2m -n kube-system Name: dashboard-admin-token-wrdvz Namespace: kube-system Labels: <none> Annotations: kubernetes.io/service-account.name: dashboard-admin kubernetes.io/service-account.uid: 4c593334-7f68-4190-85a2-f92fcab234d4 Type: kubernetes.io/service-account-token Data ==== namespace: 11 bytes token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4ta3pmMm0iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNzJjMzc1NjMtZGVkNS00NGJjLWJjOWYtYmVkMzY2NzY0Y2ZmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmUtc3lzdGVtOmRhc2hib2FyZC1hZG1pbiJ9.YxTfrsJVnE0vrDRVXFEoomsN76v5l_mGA_-n0Zi0ADIJnqOvf2h34yHE2yGXphmz6qSWsDMRgPh7uk94wpSJexIZewvN8GpVDZn8xV7tBAsW_G6Ipe6ZAeXNCynfYMXPlR1Br-PW1oC7-7-aN1QPVwhEOcAQHv5EOqJ0fopPNfbh1WxsGyQCGS8Q3jBtuTI_Huwd4rKjAOaO6271uL74N2smiWD86DLXd-LVH0Qhom16Pbj-GTkJkxyiRlR-2Q8g9hsbGJNoOKVe9gv60qHDKAf8k51rwXdRoH4ZOJqveXMXNt8WQnwlexJqNY_unL4SydqdLF4_lYnPOla5atOf2g ca.crt: 1025 bytes
上面查看到的token 后面的一大串就是，把这一串复制到页面中就可以登录。
注：实验时由于重启过一次service，导致了端口由32211变成了31718
2. kubeconfig 文件认证
创建测试一个权限小一点的，只能访问default名称空间资源的账户
创建一个 serviceAccount
[root@master pki]# kubectl create serviceaccount def-ns-admin -n default serviceaccount/def-ns-admin created [root@master pki]# kubectl describe sa/def-ns-admin -n default Name: def-ns-admin Namespace: default Labels: <none> Annotations: <none> Image pull secrets: <none> Mountable secrets: def-ns-admin-token-qfrqj Tokens: def-ns-admin-token-qfrqj Events: <none>
创建 rolebinding 绑定 def-ns-admin
[root@master pki]# kubectl create rolebinding def-ns-admin --clusterrole=admin --serviceaccount=default:def-ns-admin rolebinding.rbac.authorization.k8s.io/def-ns-admin created [root@master pki]# kubectl describe rolebinding/def-ns-admin -n default Name: def-ns-admin Labels: <none> Annotations: <none> Role: Kind: ClusterRole Name: admin Subjects: Kind Name Namespace ---- ---- --------- ServiceAccount def-ns-admin default
提示：这里和上面一样，到此步骤系统已经自动创建出一个secret，详细信息中包含token，可以直接使用此token登录dashboard，只能管理default名称空间下的资源。
创建集群和创建kubeconfig配置文件
[root@master pki]# [root@master pki]# kubectl config set-cluster kubernetes --certificate-authority=./ca.crt --server="https://172.27.1.241:6443" --embed-certs=true --kubeconfig=/root/def-ns-admin.conf Cluster "kubernetes" set. [root@master pki]# kubectl config view --kubeconfig=/root/def-ns-admin.conf apiVersion: v1 clusters: - cluster: certificate-authority-data: DATA+OMITTED server: https://172.27.1.241:6443 name: kubernetes contexts: [] current-context: "" kind: Config preferences: {} users: [] 拿到 def-ns-admin 用户的base64格式的token
先拿到对应的token后base64 解码
[root@master pki]# kubectl get secret def-ns-admin-token-rc4cz -o jsonpath={.data.token} | base64 -d eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZi1ucy1hZG1pbi10b2tlbi1yYzRjeiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJkZWYtbnMtYWRtaW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhMTcxYWQ5Ny1hNzIzLTQwN2UtYjU1ZS00YTg5ODVkNjE2YzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkZWYtbnMtYWRtaW4ifQ.GIgqCeg9a8PCn6b3GKVV7P-k8CiaqVSlg1hGcFvSAX_Xy6pkgP7WpC28HpXsVBSYjXv2G8vBJV_dTYANatmaqdg45n1DpwTrRbZIR22ks9UYXsZYwePHcoQ-i3aHpn4x6iKLdR_huYExTWXIjNdNXSz5k1yELBty0eR9172PBrSXY1X8-vj2czaB_z4JYk15mFM1-hyhDhFl_4l_gDUJanFi5MATQGTGbGePpYB_b7WXoOXPBIxN58OTiumNQLGj-5l7Pf6zX6IR8kxx4knoGQjV9BKsF4AOIFUQRKNRXUYgzG13_Qm9msah7mdSUeLgWSelgVMK 这样就获取到解码后的值了，然后保存到变量中
[root@master pki]# DEF_NS_ADMIN_TOKEN=$(kubectl get secret def-ns-admin-token-rc4cz -o jsonpath={.data.token} | base64 -d) 这样执行后就保存下来了
设置 set-credentials 保存至 def-ns-admin.conf
[root@master pki]# kubectl config set-credentials def-ns-admin --token=${DEF_NS_ADMIN_TOKEN} --kubeconfig=/root/def-ns-admin.conf [root@master pki]# kubectl config view --kubeconfig=/root/def-ns-admin.conf apiVersion: v1 clusters: - cluster: certificate-authority-data: DATA+OMITTED server: https://172.27.1.241:6443 name: kubernetes contexts: [] current-context: "" kind: Config preferences: {} users: - name: def-ns-admin # 认证的serviceaccount user: token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZi1ucy1hZG1pbi10b2tlbi1yYzRjeiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJkZWYtbnMtYWRtaW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhMTcxYWQ5Ny1hNzIzLTQwN2UtYjU1ZS00YTg5ODVkNjE2YzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkZWYtbnMtYWRtaW4ifQ.GIgqCeg9a8PCn6b3GKVV7P-k8CiaqVSlg1hGcFvSAX_Xy6pkgP7WpC28HpXsVBSYjXv2G8vBJV_dTYANatmaqdg45n1DpwTrRbZIR22ks9UYXsZYwePHcoQ-i3aHpn4x6iKLdR_huYExTWXIjNdNXSz5k1yELBty0eR9172PBrSXY1X8-vj2czaB_z4JYk15mFM1-hyhDhFl_4l_gDUJanFi5MATQGTGbGePpYB_b7WXoOXPBIxN58OTiumNQLGj-5l7Pf6zX6IR8kxx4knoGQjV9BKsF4AOIFUQRKNRXUYgzG13_Qm9msah7mdSUeLgWSelgVMKP7xSqGKeF0QHlw 配置成功
配置 def-ns-admin.conf 上下文
[root@master pki]# kubectl config set-context def-ns-admin@kubernetes --cluster=kubernetes --user=def-ns-admin --kubeconfig=/root/def-ns-admin.conf Context "def-ns-admin@kubernetes" created. [root@master pki]# kubectl config view --kubeconfig=/root/def-ns-admin.conf apiVersion: v1 clusters: - cluster: # 这里是上面创建的集群 certificate-authority-data: DATA+OMITTED server: https://172.27.1.241:6443 # 这里是集群地址 name: kubernetes # 这里集群名称 contexts: - context: # 上面创建的上下文 cluster: kubernetes # 集群 user: def-ns-admin # serviceaccount name: def-ns-admin@kubernetes current-context: "" # 这里还没有配置当前上下文 kind: Config preferences: {} users: - name: def-ns-admin user: token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZi1ucy1hZG1pbi10b2tlbi1yYzRjeiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJkZWYtbnMtYWRtaW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhMTcxYWQ5Ny1hNzIzLTQwN2UtYjU1ZS00YTg5ODVkNjE2YzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkZWYtbnMtYWRtaW4ifQ.GIgqCeg9a8PCn6b3GKVV7P-k8CiaqVSlg1hGcFvSAX_Xy6pkgP7WpC28HpXsVBSYjXv2G8vBJV_dTYANatmaqdg45n1DpwTrRbZIR22ks9UYXsZYwePHcoQ-i3aHpn4x6iKLdR_huYExTWXIjNdNXSz5k1yELBty0eR9172PBrSXY1X8-vj2czaB_z4JYk15mFM1-hyhDhFl_4l_gDUJanFi5MATQGTGbGePpYB_b7WXoOXPBIxN58OTiumNQLGj-5l7Pf6zX6IR8kxx4knoGQjV9BKsF4AOIFUQRKNRXUYgzG13_Qm9msah7mdSUeLgWSelgVMKP7xSqGKeF0QHlw 配置当前上下文
[root@master pki]# kubectl config view --kubeconfig=/root/def-ns-admin.conf apiVersion: v1 clusters: - cluster: certificate-authority-data: DATA+OMITTED server: https://172.27.1.241:6443 name: kubernetes contexts: - context: cluster: kubernetes user: def-ns-admin name: def-ns-admin@kubernetes current-context: def-ns-admin@kubernetes # 当前上下文用户 kind: Config preferences: {} users: - name: def-ns-admin user: token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImRlZi1ucy1hZG1pbi10b2tlbi1yYzRjeiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50Lm5hbWUiOiJkZWYtbnMtYWRtaW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJhMTcxYWQ5Ny1hNzIzLTQwN2UtYjU1ZS00YTg5ODVkNjE2YzQiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGVmYXVsdDpkZWYtbnMtYWRtaW4ifQ.GIgqCeg9a8PCn6b3GKVV7P-k8CiaqVSlg1hGcFvSAX_Xy6pkgP7WpC28HpXsVBSYjXv2G8vBJV_dTYANatmaqdg45n1DpwTrRbZIR22ks9UYXsZYwePHcoQ-i3aHpn4x6iKLdR_huYExTWXIjNdNXSz5k1yELBty0eR9172PBrSXY1X8-vj2czaB_z4JYk15mFM1-hyhDhFl_4l_gDUJanFi5MATQGTGbGePpYB_b7WXoOXPBIxN58OTiumNQLGj-5l7Pf6zX6IR8kxx4knoGQjV9BKsF4AOIFUQRKNRXUYgzG13_Qm9msah7mdSUeLgWSelgVMKP7xSqGKeF0QHlw 这样，就把当前 上下文的用户配置成功
测试使用kubeconfig文件登陆
下载刚刚生成好的 def-ns-admin.conf
然后测试登陆:
基于 tls 验证
创建 dashboard 专用的 tls 证书
进入到 kubernetes 集群的 master 节点中的 /etc/kubernetes/pki/ 目录
这个目录是专门存放 kubernetes 集群证书的目录
生成dashboard 用户的私钥
[root@master ~]# cd /etc/kubernetes/pki/ [root@master pki]# (umask 077; openssl genrsa -out dashboard.key 2048) Generating RSA private key, 2048 bit long modulus ........................+++ .........................................................+++ e is 65537 (0x10001) [root@master pki]# ll dashboard.key -rw------- 1 root root 1679 Aug 22 17:51 dashboard.key
生成证书签署请求
[root@master pki]# openssl req -new -key dashboard.key -out dashboard.csr -subj "/O=tracy/CN=dashboard" [root@master pki]# ll dashboard.csr -rw-r--r-- 1 root root 911 Aug 22 17:54 dashboard.csr
给dashboard证书签证
这里需要注意，一定要kubernetes集群中的 ca.crt 和 ca.key 来签署，这样才能在此集群中使用。
[root@master pki]# openssl x509 -req -in dashboard.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out dashboard.crt -days 365 Signature ok subject=/O=tracy/CN=dashboard Getting CA Private Key
这样就签署完成了
创建 dashboard 对应的 secret
[root@master pki]# kubectl create secret generic dashboard-cert -n kube-system --from-file=dashboard.crt --from-file=dashboard.key secret/dashboard-cert created [root@master pki]# kubectl get secret -n kube-system dashboard-cert NAME TYPE DATA AGE dashboard-cert Opaque 2 16s [root@master pki]# kubectl describe secret -n kube-system dashboard-cert Name: dashboard-cert Namespace: kube-system Labels: <none> Annotations: <none> Type: Opaque Data ==== dashboard.crt: 1001 bytes dashboard.key: 1679 bytes
Dashboard部署总结：
1、部署：
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-
flannel.yml
2、将Service改为NodePort
kubectl patch svc kubernetes-dashboard -p '{"spec":{"type":"NodePort"}}' -n kube-system
3、认证：
认证时的账号必须为ServiceAccount：被dashboard pod拿来由kubernetes进行认证；
token:
（1）创建ServiceAccount，根据其管理目标，使用rolebinding或clusterrolebinding绑定至合理role或
clusterrole;
（2）获取到此ServiceAccount的secret，查看secret的详细信息，其中就有token；
kubeconfig: 把ServiceAccount的token封装为kubeconfig文件
（1）创建ServiceAccount，根据其管理目标，使用rolebinding或clusterrolebinding绑定至合理role或
clusterrole;
（2）kubectl get secret | awk '/^ServiceAccount/{print $1}'
KUBE_TOKEN=$(kubectl get secret SERVCIEACCOUNT_SERRET_NAME -o jsonpath={.data.token} |
base64 -d)
（3）生成kubeconfig文件
kubectl config set-cluster --kubeconfig=/PATH/TO/SOMEFILE
kubectl config set-credentials NAME --token=$KUBE_TOKEN --kubeconfig=/PATH/TO/SOMEFILE
kubectl config set-context
kubectl config use-context
kubernetes集群的管理方式：
1、命令式：create, run, expose, delete, edit, ...
2、命令式配置文件: create -f /PATH/TO/RESOURCE_CONFIGURATION_FILE, delete -f, replace -f
3、声明式配置文件: apply -f, patch,