网上讲述如何部署k8s集群的文章很多，特别是k8s中文社区里面，每个平台的部署方式都有详细说明。但就是看了中文社区的指导，发现一路是坑，第一个源访问的时候就404 NotFound， 更别说那生涩的翻译和版本匹配问题。
如此一来还不如自己写一个。
1. 开虚机
系统环境CentOS 7.2 . 这方面不再赘述
2. 设置/etc/hosts
简单来说就是把master和minion主机都用域名在hosts文件中记录一下。
3. 集群主机都安装kubernetes和etcd
直接yum install . 安装kubernetes 时会顺带把docker等都安装上。 此文章编写时，阿里云的kubernetes版本为1.5.2.
这里还需要注意：应该在etcdctl中配置flannel，否则flannel无法正常启动。
[root@k8s-master home]# etcdctl set /flannel/network/config '{ "Network": "172.16.0.0/16", "SubnetLen": 24, "Backend": { "Type": "vxlan" } }'
4. 配置apiserver
按照中文社区的指导配置（但最终并没有使用该配置，看下去就知道）
修改/etc/kubernetes/apiserver
修改/etc/kubernetes/config
由于这条路不通，具体修改内容就不贴了。
5. master配置启动脚本
#/bin/bashfor SERVICES in etcd kube-apiserver kube-controller-manager kube-scheduler; dosystemctl restart $SERVICESsystemctl enable $SERVICESsystemctl status $SERVICESdone
走到这里，按照中文社区的指导，应该就能直接起来了。但其实并不能！！！
收到以下错误：
Sep 29 17:06:15 debug010000002015 kube-apiserver: W0929 17:06:15.881473 21259 handlers.go:50] Authentication is disabledSep 29 17:06:15 debug010000002015 kube-apiserver: [restful] 2018/09/29 17:06:15 log.go:30: [restful/swagger] listing is available at https://172.16.7.93:6443/swaggerapi/Sep 29 17:06:15 debug010000002015 kube-apiserver: [restful] 2018/09/29 17:06:15 log.go:30: [restful/swagger] https://172.16.7.93:6443/swaggerui/ is mapped to folder /swagger-ui/Sep 29 17:06:15 debug010000002015 kube-apiserver: E0929 17:06:15.984071 21259 reflector.go:199] k8s.io/kubernetes/plugin/pkg/admission/resourcequota/resource_access.go:83: Failed to list *api.ResourceQuota: Get http://127.0.0.1:18080/api/v1/resourcequotas?resourceVersion=0: dial tcp 127.0.0.1:18080: getsockopt: connection refusedSep 29 17:06:15 debug010000002015 kube-apiserver: E0929 17:06:15.984217 21259 reflector.go:199] pkg/controller/informers/factory.go:89: Failed to list *api.Namespace: Get http://127.0.0.1:18080/api/v1/namespaces?resourceVersion=0: dial tcp 127.0.0.1:18080: getsockopt: connection refusedSep 29 17:06:15 debug010000002015 kube-apiserver: E0929 17:06:15.987986 21259 reflector.go:199] pkg/controller/informers/factory.go:89: Failed to list *api.LimitRange: Get http://127.0.0.1:18080/api/v1/limitranges?resourceVersion=0: dial tcp 127.0.0.1:18080: getsockopt: connection refusedSep 29 17:06:16 debug010000002015 kube-apiserver: F0929 17:06:16.058072 21259 genericapiserver.go:189] unable to load server certificate: open /var/run/kubernetes/apiserver.key: permission deniedSep 29 17:06:16 debug010000002015 systemd: kube-apiserver.service: main process exited, code=exited, status=255/n/aSep 29 17:06:16 debug010000002015 systemd: Failed to start Kubernetes API Server.Sep 29 17:06:16 debug010000002015 systemd: Unit kube-apiserver.service entered failed state.Sep 29 17:06:16 debug010000002015 systemd: kube-apiserver.service failed.Sep 29 17:06:16 debug010000002015 systemd: kube-apiserver.service holdoff time over, scheduling restart.
查询了Google和Baidu都无果。
但是测试发现直接用命令行启动kube-api 是成功的。因此只好采取直接修改systemctl service文件的做法。
修改kube-apiserver.service的启动脚本， 路径是 /lib/systemd/system/kube-apiserver.service
内容如下：
[root@k8s-master home]# vi /lib/systemd/system/kube-apiserver.service [Unit]Description=Kubernetes API ServiceDocumentation=https://github.com/GoogleCloudPlatform/kubernetesAfter=network.targetAfter=etcd.service [Service]EnvironmentFile=-/etc/kubernetes/configEnvironmentFile=-/etc/kubernetes/apiserver#ExecStart=/usr/bin/kube-apiserver \# $KUBE_LOGTOSTDERR \# $KUBE_LOG_LEVEL \# $KUBE_ETCD_SERVERS \# $KUBE_API_ADDRESS \# $KUBE_API_PORT \# $KUBELET_PORT \# $KUBE_ALLOW_PRIV \# $KUBE_SERVICE_ADDRESSES \# $KUBE_ADMISSION_CONTROL \# $KUBE_API_ARGS ExecStart=/usr/bin/kube-apiserver --allow_privileged=true --logtostderr=false --v=6 --log-dir=/var/log/k8s/kube-apiserver --insecure-bind-address=0.0.0.0 --insecure-port=8080 --admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota,ServiceAccount,AlwaysPullImages,SecurityContextDeny --etcd_servers=http://x.x.x.x:2379 --master-service-namespace=master --secure-port=6443 --bind-address=0.0.0.0 --service-cluster-ip-range=10.0.0.0/16 --max-requests-inflight=1000 --storage-backend=etcd3 --tls-cert-file=/etc/kubernetes/pki/apiserver.pem \ --tls-private-key-file=/etc/kubernetes/pki/apiserver-key.pem \ --client-ca-file=/etc/kubernetes/pki/ca.pem \ --service-account-key-file=/etc/kubernetes/pki/ca-key.pemKillMode=control-groupRestart=on-failureRestartSec=10 [Install]WantedBy=multi-user.target
其中 ： --etcd_servers=http://xx.xx.xx.xx:2379 为本机eth0网卡IP,需要替换。
相关ssl文件需要用openssl自己生成。或者使用insecure模式。
将 etcd的配置文件，路径如下：/etc/etcd/etcd.conf ， 其中下述内容由监听本地回环改为监听0.0.0.0
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"
修改flannel.service文件
[root@k8s-master home]# vi /lib/systemd/system/flanneld.service [Unit]Description=Flanneld overlay address etcd agentAfter=network.targetAfter=network-online.targetWants=network-online.targetAfter=etcd.serviceBefore=docker.service [Service]Type=notifyEnvironmentFile=/etc/sysconfig/flanneldEnvironmentFile=-/etc/sysconfig/docker-networkExecStart=/usr/bin/flanneld -etcd-endpoints=http://x.x.x.x:2379 -etcd-prefix=/flannel/network -iface=eth0#ExecStart=/usr/bin/flanneld-start $FLANNEL_OPTIONS#ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/dockerRestart=on-failure [Install]WantedBy=multi-user.targetWantedBy=docker.service
其中 ： --etcd_servers=http://xx.xx.xx.xx:2379 为本机eth0网卡IP,需要替换。
修改kube-controller-manager.service文件
[root@k8s-master home]# vi /lib/systemd/system/kube-controller-manager.service Description=Kubernetes Controller ManagerDocumentation=https://github.com/GoogleCloudPlatform/kubernetes [Service]EnvironmentFile=-/etc/kubernetes/configEnvironmentFile=-/etc/kubernetes/controller-manager#ExecStart=/usr/bin/kube-controller-manager \# $KUBE_LOGTOSTDERR \# $KUBE_LOG_LEVEL \# $KUBE_MASTER \# $KUBE_CONTROLLER_MANAGER_ARGS ExecStart=/usr/bin/kube-controller-manager --logtostderr=false --v=6 --log-dir=/var/log/k8s/kube-controller-manager --namespace-sync-period=5m0s --node-monitor-grace-period=40s --node-monitor-period=5s --node-startup-grace-period=1m0s --node-sync-period=10s --pod-eviction-timeout=5m0s --pvclaimbinder-sync-period=10s --register-retry-count=20 --kubeconfig=/etc/kubernetes/controller-manager.conf \ --cluster-name=kubernetes \ --service-cluster-ip-range=10.0.0.0/16 \ --cluster-signing-cert-file=/etc/kubernetes/pki/ca.pem \ --cluster-signing-key-file=/etc/kubernetes/pki/ca-key.pem \ --service-account-private-key-file=/etc/kubernetes/pki/ca-key.pem \ --root-ca-file=/etc/kubernetes/pki/ca.pem Restart=on-failureLimitNOFILE=65536 [Install]WantedBy=multi-user.target
修改kube-scheduler.service文件
[root@k8s-master home]# vi /lib/systemd/system/kube-scheduler.service [Unit]Description=Kubernetes Scheduler PluginDocumentation=https://github.com/GoogleCloudPlatform/kubernetes [Service]EnvironmentFile=-/etc/kubernetes/configEnvironmentFile=-/etc/kubernetes/scheduler#ExecStart=/usr/bin/kube-scheduler \# $KUBE_LOGTOSTDERR \# $KUBE_LOG_LEVEL \# $KUBE_MASTER \# $KUBE_SCHEDULER_ARGS ExecStart=/usr/bin/kube-scheduler --logtostderr=false --v=6 --log-dir=/var/log/k8s/kube-scheduler --algorithm-provider=DefaultProvider --kubeconfig=/etc/kubernetes/scheduler.conf Restart=on-failureLimitNOFILE=65536 [Install]WantedBy=multi-user.target
修改kube-proxy.service文件
[root@k8s-master home]# vi /lib/systemd/system/kube-proxy.service [Unit]Description=Kubernetes Kube-Proxy ServerDocumentation=https://github.com/GoogleCloudPlatform/kubernetesAfter=network.target [Service]EnvironmentFile=-/etc/kubernetes/configEnvironmentFile=-/etc/kubernetes/proxy#ExecStart=/usr/bin/kube-proxy \# $KUBE_LOGTOSTDERR \# $KUBE_LOG_LEVEL \# $KUBE_MASTER \# $KUBE_PROXY_ARGS ExecStart=/usr/bin/kube-proxy --master=http://x.x.x.x:8080 --hostname-override=k8s-master --proxy-mode=iptables -v=6 --logtostderr=false --log-dir=/var/log/k8s/kube-proxyRestart=on-failureLimitNOFILE=65536 [Install]WantedBy=multi-user.target
其中 xx.xx.xx.xx:8080 为本机eth0网卡IP,需要替换。
修改kubelet.service文件
[root@k8s-master home]# vi /lib/systemd/system/kubelet.service [Unit]Description=Kubernetes Kubelet ServerDocumentation=https://github.com/GoogleCloudPlatform/kubernetesAfter=docker.serviceRequires=docker.service [Service]WorkingDirectory=/var/lib/kubeletEnvironmentFile=-/etc/kubernetes/configEnvironmentFile=-/etc/kubernetes/kubelet#ExecStart=/usr/bin/kubelet \# $KUBE_LOGTOSTDERR \# $KUBE_LOG_LEVEL \# $KUBELET_API_SERVER \# $KUBELET_ADDRESS \# $KUBELET_PORT \# $KUBELET_HOSTNAME \# $KUBE_ALLOW_PRIV \# $KUBELET_POD_INFRA_CONTAINER \# $KUBELET_ARGS ExecStart=/usr/bin/kubelet --allow-privileged=true \ --logtostderr=false \ --v=6 \ --log-dir=/var/log/k8s/kubelet \ --address=x.x.x.x \ --cluster-dns=10.0.1.10 \ --hostname-override=k8s-master \ --cluster-domain=cluster.local \ --kubeconfig=/etc/kubernetes/kubelet.conf \ --pod-manifest-path=/etc/kubernetes/manifest \ --allow-privileged=true \ --authorization-mode=AlwaysAllow \ --fail-swap-on=false \ --cgroup-driver=systemd \ --pod-infra-container-image=registry.aliyuncs.com/archon/pause-amd64:3.0 Restart=on-failure [Install]WantedBy=multi-user.target
其中 xx.xx.xx.xx 为本机eth0网卡IP,需要替换。
registry.aliyuncs.com/archon/pause-amd64:3.0 来源自https://segmentfault.com/q/1010000008763165/a-1020000008824481
完毕后，再运行启动脚本，此时所有组件都能够正常启动。
6. minion配置文件
路径为： /etc/kubernetes/kubelet 以及 /etc/kubernetes/config
config文件内容如下：
#### kubernetes system config## The following values are used to configure various aspects of all# kubernetes services, including## kube-apiserver.service# kube-controller-manager.service# kube-scheduler.service# kubelet.service# kube-proxy.service# logging to stderr means we get it in the systemd journalKUBE_LOGTOSTDERR="--logtostderr=true" # journal message level, 0 is debugKUBE_LOG_LEVEL="--v=0" # Should this cluster be allowed to run privileged docker containersKUBE_ALLOW_PRIV="--allow-privileged=false" # How the controller-manager, scheduler, and proxy find the apiserverKUBE_MASTER="--master=http://k8s-master:8080" # Comma separated list of nodes in the etcd clusterKUBE_ETCD_SERVERS=”–etcd_servers=http://k8s-master:4001″ # logging to stderr means we get it in the systemd journal
kubelet文件如下：
#### kubernetes kubelet (minion) config # The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)KUBELET_ADDRESS="--address=0.0.0.0" # The port for the info server to serve on# KUBELET_PORT="--port=10250" # You may leave this blank to use the actual hostnameKUBELET_HOSTNAME="--hostname-override=k8s-slave" # location of the api-serverKUBELET_API_SERVER="--api-servers=http://k8s-master:8080" # pod infrastructure containerKUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest" # Add your own!KUBELET_ARGS=""
验证服务状态：
[root@k8s-slave home]# kubectl get csNAME STATUS MESSAGE ERRORscheduler Healthy ok controller-manager Healthy ok etcd-0 Healthy {"health": "true"}
6. minion配置启动脚本
修改kube-proxy.service文件
[root@k8s-slave home]# vi /lib/systemd/system/kube-proxy.service [Unit]Description=Kubernetes Kube-Proxy ServerDocumentation=https://github.com/GoogleCloudPlatform/kubernetesAfter=network.target [Service]EnvironmentFile=-/etc/kubernetes/configEnvironmentFile=-/etc/kubernetes/proxy#ExecStart=/usr/bin/kube-proxy \# $KUBE_LOGTOSTDERR \# $KUBE_LOG_LEVEL \# $KUBE_MASTER \# $KUBE_PROXY_ARGS ExecStart=/usr/bin/kube-proxy --master=http://x.x.x.x:8080 --hostname-override=k8s-slave --proxy-mode=iptables -v=6 --logtostderr=false --log-dir=/var/log/k8s/kube-proxy Restart=on-failureLimitNOFILE=65536 [Install]WantedBy=multi-user.target
其中 xx.xx.xx.xx 为master eth0网卡IP,需要替换。
修改kubelet配置文件
[root@k8s-slave home]# vi /lib/systemd/system/kubelet.service [Unit]Description=Kubernetes Kubelet ServerDocumentation=https://github.com/GoogleCloudPlatform/kubernetesAfter=docker.serviceRequires=docker.service [Service]WorkingDirectory=/var/lib/kubeletEnvironmentFile=-/etc/kubernetes/configEnvironmentFile=-/etc/kubernetes/kubelet#ExecStart=/usr/bin/kubelet \# $KUBE_LOGTOSTDERR \# $KUBE_LOG_LEVEL \# $KUBELET_API_SERVER \# $KUBELET_ADDRESS \# $KUBELET_PORT \# $KUBELET_HOSTNAME \# $KUBE_ALLOW_PRIV \# $KUBELET_POD_INFRA_CONTAINER \# $KUBELET_ARGS ExecStart=/usr/bin/kubelet --allow-privileged=true \ --logtostderr=false \ --v=6 \ --log-dir=/var/log/k8s/kubelet \ --address=0.0.0.0 \ --cluster-dns=10.0.1.10 \ --hostname-override=k8s-slave \ --cluster-domain=cluster.local \ --kubeconfig=/etc/kubernetes/kubelet.conf \ --pod-manifest-path=/etc/kubernetes/manifest \ --allow-privileged=true \ --authorization-mode=AlwaysAllow \ --fail-swap-on=false \ --cgroup-driver=systemd \ --pod-infra-container-image=registry.aliyuncs.com/archon/pause-amd64:3.0 Restart=on-failure [Install]WantedBy=multi-user.target
在etc/profile文件末尾添加以下内容：
export KUBERNETES_MASTER=http://x.x.x.x:8080
其中 xx.xx.xx.xx 为master eth0网卡IP,需要替换。
for SERVICES in kube-proxy kubelet docker; dosystemctl restart $SERVICESsystemctl enable $SERVICESsystemctl status $SERVICESdone
运行启动脚本，此时所有服务可以正常启动。
验证服务状态：
[root@k8s-slave home]# kubectl get nodesNAME STATUS ROLES AGE VERSIONk8s-master Ready <none> 3h v1.9.0k8s-slave Ready <none> 2h v1.9.0
至此集群部署完毕。
来源：oschina
链接：https://my.oschina.net/u/4562984/blog/4881889