环境说明
三台主机：
k8s-master 10.0.3.225
k8s-node1 10.0.3.226
k8s-node2 10.0.3.227
配置主机名解析
[root@k8s-master kubernetes]# vim /etc/hosts
10.0.3.225 k8s-master
10.0.3.226 k8s-node1
10.0.3.227 k8s-node2
关闭防火墙和SElinux
[root@k8s-master kubernetes]# systemctl stop firewalld
[root@k8s-master kubernetes]# getenforce
Disabled
设置环境变量
vim /root/.bash_profile
PATH=$PATH:$HOME/bin:/opt/kubernetes/bin
source /root/.bash_profile
为了拷贝方便，设置SSH免密
[root@k8s-master src]# ssh-keygen
[root@k8s-master src]# ssh-copy-id 10.0.3.225
[root@k8s-master src]# ssh-copy-id 10.0.3.226
[root@k8s-master src]# ssh-copy-id 10.0.3.227
系统环境初始化
1.安装Docker（三台操作）
第一步：使用国内Docker源
[root@linux-node1 ~]# cd /etc/yum.repos.d/
[root@linux-node1 yum.repos.d]# wget \
https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
第二步：Docker安装：
[root@linux-node1 ~]# yum install -y docker-ce
第三步：启动后台进程：
[root@linux-node1 ~]# systemctl start docker
2.准备部署目录（三台操作）
mkdir -p /opt/kubernetes/{cfg,bin,ssl,log}
3.准备软件包百度网盘下载地址：（在mastrt操作）
[https://pan.baidu.com/s/1zs8sCouDeCQJ9lghH1BPiw](https://pan.baidu.com/s/1zs8sCouDeCQJ9lghH1BPiw)
cd /usr/local/
unzip k8s-v1.10.1-manual.zip
cd k8s-v1.10.1-manual/k8s-v1.10.1/
mv * /usr/local/src/ k8s官网地址：https://github.com/kubernetes
4.解压软件包（在mastrt操作）
cd /usr/local/src/
tar zxvf kubernetes.tar.gz && \
tar zxvf kubernetes-server-linux-amd64.tar.gz && \
tar zxvf kubernetes-client-linux-amd64.tar.gz && \
tar zxvf kubernetes-node-linux-amd64.tar.gz
所有软件包都在kubernetes目录下了
[root@k8s-master src]# ls
cfssl-certinfo_linux-amd64 cfssl_linux-amd64 etcd-v3.2.18-linux-amd64.tar.gz kubernetes kubernetes-node-linux-amd64.tar.gz kubernetes.tar.gz
cfssljson_linux-amd64 cni-plugins-amd64-v0.7.1.tgz flannel-v0.10.0-linux-amd64.tar.gz kubernetes-client-linux-amd64.tar.gz kubernetes-server-linux-amd64.tar.gz
[root@k8s-master src]# cd kubernetes/
[root@k8s-master kubernetes]# ls
addons client cluster docs examples hack kubernetes-src.tar.gz LICENSES node README.md server third_party version
[root@k8s-master kubernetes]#
手动制作CA证书
1.安装 CFSSL
上面的安装包，已经下载好cfssl了。移动/opt/kubernetes/bin/即可。
cd /usr/local/src
chmod +x cfssl* && \
mv cfssl-certinfo_linux-amd64 /opt/kubernetes/bin/cfssl-certinfo && \
mv cfssljson_linux-amd64 /opt/kubernetes/bin/cfssljson && \
mv cfssl_linux-amd64 /opt/kubernetes/bin/cfssl
复制cfssl命令文件到k8s-node1和k8s-node2节点。如果实际中多个节点，就都需要同步复制。
[root@k8s-master src]# scp /opt/kubernetes/bin/cfssl* 10.0.3.226:/opt/kubernetes/bin/
[root@k8s-master src]# scp /opt/kubernetes/bin/cfssl* 10.0.3.227:/opt/kubernetes/bin/
CFSSL下载地址：
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
2.初始化cfssl
[root@k8s-master src]# cd /usr/local/src/
[root@k8s-master src]# mkdir ssl && cd ssl
[root@k8s-master ssl]# cfssl print-defaults config > config.json
[root@k8s-master ssl]# cfssl print-defaults csr > csr.json
3.创建用来生成 CA 文件的 JSON 配置文件
[root@k8s-master ssl]# vim ca-config.json
{
"signing": {
"default": {
"expiry": "8760h"
},
"profiles": {
"kubernetes": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "8760h"
}
}
}
}
4.创建用来生成 CA 证书签名请求（CSR）的 JSON 配置文件
[root@k8s-master ssl]# vim ca-csr.json
{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
5.生成CA证书（ca.pem）和密钥（ca-key.pem）
[root@k8s-master ssl]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca
2018/11/13 16:12:11 [INFO] generating a new CA key and certificate from CSR
2018/11/13 16:12:11 [INFO] generate received request
2018/11/13 16:12:11 [INFO] received CSR
2018/11/13 16:12:11 [INFO] generating key: rsa-2048
2018/11/13 16:12:12 [INFO] encoded CSR
2018/11/13 16:12:12 [INFO] signed certificate with serial number 672169645465776774912654230107317469678160608506
[root@k8s-master ssl]# ls -l ca*
-rw-r--r-- 1 root root 290 Nov 13 16:10 ca-config.json
-rw-r--r-- 1 root root 1001 Nov 13 16:12 ca.csr
-rw-r--r-- 1 root root 208 Nov 13 16:11 ca-csr.json
-rw------- 1 root root 1675 Nov 13 16:12 ca-key.pem
-rw-r--r-- 1 root root 1359 Nov 13 16:12 ca.pem
6.分发证书
[root@k8s-master ssl]# cp ca.csr ca.pem ca-key.pem ca-config.json /opt/kubernetes/ssl
#scp证书到k8s-node1和k8s-node2节点
[root@k8s-master ssl]# scp ca.csr ca.pem ca-key.pem ca-config.json 10.0.3.226:/opt/kubernetes/ssl
ca.csr 100% 1001 1.0KB/s 00:00
ca.pem 100% 1359 1.3KB/s 00:00
ca-key.pem 100% 1675 1.6KB/s 00:00
ca-config.json 100% 290 0.3KB/s 00:00
[root@k8s-master ssl]# scp ca.csr ca.pem ca-key.pem ca-config.json 10.0.3.227:/opt/kubernetes/ssl
ca.csr 100% 1001 1.0KB/s 00:00
ca.pem 100% 1359 1.3KB/s 00:00
ca-key.pem 100% 1675 1.6KB/s 00:00
ca-config.json 100% 290 0.3KB/s 00:00
部署ETCD集群
详细介绍查看etcd官网： https://github.com/coreos/etcd-operator
0.准备etcd软件包
[root@k8s-master src]# pwd
/usr/local/src
[root@k8s-master src]# tar zxvf etcd-v3.2.18-linux-amd64.tar.gz
[root@k8s-master src]# cd etcd-v3.2.18-linux-amd64/
[root@k8s-master etcd-v3.2.18-linux-amd64]# ls
Documentation etcd etcdctl README-etcdctl.md README.md READMEv2-etcdctl.md
[root@k8s-master etcd-v3.2.18-linux-amd64]# cp etcd etcdctl /opt/kubernetes/bin/
[root@k8s-master etcd-v3.2.18-linux-amd64]# cp etcd etcdctl 10.0.3.226:/opt/kubernetes/bin/
[root@k8s-master etcd-v3.2.18-linux-amd64]# scp etcd etcdctl 10.0.3.227:/opt/kubernetes/bin/下载地址：wget https://github.com/coreos/etcd/releases/download/v3.2.18/etcd-v3.2.18-linux-amd64.tar.gz
1.创建 etcd 证书签名请求：
[root@k8s-master ssl]# cd /usr/local/src/ssl/
[root@k8s-master ssl]# vim etcd-csr.json
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
"10.0.3.225", #指定etcd节点的IP地址
"10.0.3.226",
"10.0.3.227"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "BeiJing",
"L": "BeiJing",
"O": "k8s",
"OU": "System"
}
]
}
2.生成 etcd 证书和私钥：
[root@k8s-master ssl]# cfssl gencert -ca=/opt/kubernetes/ssl/ca.pem \
> -ca-key=/opt/kubernetes/ssl/ca-key.pem \
> -config=/opt/kubernetes/ssl/ca-config.json \
> -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
会生成一下证书文件
[root@k8s-master ssl]# ls -l etcd*
-rw-r--r-- 1 root root 1062 Nov 13 16:40 etcd.csr
-rw-r--r-- 1 root root 278 Nov 13 16:39 etcd-csr.json
-rw------- 1 root root 1675 Nov 13 16:40 etcd-key.pem
-rw-r--r-- 1 root root 1436 Nov 13 16:40 etcd.pem
3.将证书移动到/opt/kubernetes/ssl目录下
[root@k8s-master ssl]# cp etcd*.pem /opt/kubernetes/ssl
[root@k8s-master ssl]# scp etcd*.pem 10.0.3.226:/opt/kubernetes/ssl
etcd-key.pem 100% 1675 1.6KB/s 00:00
etcd.pem 100% 1436 1.4KB/s 00:00
[root@k8s-master ssl]# scp etcd*.pem 10.0.3.227:/opt/kubernetes/ssl
etcd-key.pem 100% 1675 1.6KB/s 00:00
etcd.pem
4.设置ETCD配置文件
[root@k8s-master ssl]# vim /opt/kubernetes/cfg/etcd.conf
#[member]
ETCD_NAME="etcd-node1" #唯一的，每个节点不能一样
ETCD_DATA_DIR="/var/lib/etcd/default.etcd" #数据存放目录
#ETCD_SNAPSHOT_COUNTER="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
ETCD_LISTEN_PEER_URLS="https://10.0.3.225:2380" #集群之间通信使用
ETCD_LISTEN_CLIENT_URLS="https://10.0.3.225:2379,https://127.0.0.1:2379" #是给客户端使用
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#[cluster]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.0.3.225:2380"
# if you use different ETCD_NAME (e.g. test),
# set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
ETCD_INITIAL_CLUSTER="etcd-node1=https://10.0.3.225:2380,etcd-node2=https://10.0.3.226:2380,etcd-node3=https://10.0.3.227:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="k8s-etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="https://10.0.3.225:2379"
#[security]
CLIENT_CERT_AUTH="true"
ETCD_CA_FILE="/opt/kubernetes/ssl/ca.pem"
ETCD_CERT_FILE="/opt/kubernetes/ssl/etcd.pem"
ETCD_KEY_FILE="/opt/kubernetes/ssl/etcd-key.pem"
PEER_CLIENT_CERT_AUTH="true"
ETCD_PEER_CA_FILE="/opt/kubernetes/ssl/ca.pem"
ETCD_PEER_CERT_FILE="/opt/kubernetes/ssl/etcd.pem"
ETCD_PEER_KEY_FILE="/opt/kubernetes/ssl/etcd-key.pem"
5.创建ETCD系统服务
[root@k8s-master ssl]# vim /etc/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
[Service]
Type=simple
WorkingDirectory=/var/lib/etcd
EnvironmentFile=-/opt/kubernetes/cfg/etcd.conf
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /opt/kubernetes/bin/etcd"
Type=notify
[Install]
WantedBy=multi-user.target
6.重新加载系统服务
[root@k8s-master ssl]# systemctl daemon-reload
[root@k8s-master ssl]# systemctl enable etcd.serviceetcd.conf和etcd.service拷贝到其他节点
[root@k8s-master ssl]# scp /opt/kubernetes/cfg/etcd.conf 10.0.3.226:/opt/kubernetes/cfg/
etcd.conf 100% 1148 1.1KB/s 00:00
[root@k8s-master ssl]# scp /opt/kubernetes/cfg/etcd.conf 10.0.3.227:/opt/kubernetes/cfg/
etcd.conf 100% 1148 1.1KB/s 00:00
[root@k8s-master ssl]# scp /etc/systemd/system/etcd.service 10.0.3.226:/opt/kubernetes/cfg/
etcd.service 100% 314 0.3KB/s 00:00
[root@k8s-master ssl]# scp /etc/systemd/system/etcd.service 10.0.3.227:/opt/kubernetes/cfg/
etcd.service 其他节点需要修改一下配置。以node1为例，node2一样要修改。
[root@k8s-node1 yum.repos.d]# vim /opt/kubernetes/cfg/etcd.conf
2 ETCD_NAME="etcd-node2"
7 ETCD_LISTEN_PEER_URLS="https://10.0.3.226:2380"
8 ETCD_LISTEN_CLIENT_URLS="https://10.0.3.226:2379,https://127.0.0.1:2379"
13 ETCD_INITIAL_ADVERTISE_PEER_URLS="https://10.0.3.226:2380"
19 ETCD_ADVERTISE_CLIENT_URLS="https://10.0.3.226:2379"
同样需要reload一下。systemctl daemon-reload
systemctl enable etcd
在所有节点上创建etcd存储目录并启动etcd
mkdir /var/lib/etcd
systemctl start etcd
systemctl status etcd
netstat -lntup|grep 23
需要在所有的 etcd 节点重复上面的步骤，直到所有机器的 etcd 服务都已启动。
7.验证集群
[root@k8s-node1 ~]# etcdctl --endpoints=https://10.0.3.225:2379 \
> --ca-file=/opt/kubernetes/ssl/ca.pem \
> --cert-file=/opt/kubernetes/ssl/etcd.pem \
> --key-file=/opt/kubernetes/ssl/etcd-key.pem cluster-health
member 981424cb437f053 is healthy: got healthy result from https://10.0.3.226:2379
member 7a30be4a7549b699 is healthy: got healthy result from https://10.0.3.227:2379
member d12a89096ab8db09 is healthy: got healthy result from https://10.0.3.225:2379
cluster is healthy
来源：oschina
链接：https://my.oschina.net/u/4257889/blog/3753612