0 背景

由于本次部署的节点有20个，2个Master，18个Node，而Calico默认采用Full-mesh BGP，将导致建立的连接数过多，故引入RR完成路由的分发

1 节点及配置规划

1.1 地址规划

ip范围	角色
192.168.2.1-2	RR
192.168.2.3-20	RR-Client

1.2 关键配置

修改/etc/ansible/roles/calico/defaults 配置

本K8S集群运行在同网段kvm虚机上，虚机间没有网络ACL限制，因此可以设置CALICO_IPV4POOL_IPIP=off，如果你的主机位于不同网段，或者运行在公有云上需要打开这个选项 CALICO_IPV4POOL_IPIP=always

# 设置 CALICO_IPV4POOL_IPIP=“off”,可以提高网络性能
CALICO_IPV4POOL_IPIP: "off"

安装完成后会发现，网卡并未像开启IPIP那样生成tunl0网卡，而是通过物理网卡获取到各节点POD网段的路由，说明配置成功；

查看路由表：

路由表一开始不一定每个节点都会宣告进来，待节点调度生成过POD后即可宣告路由，就可以看到更新后的路由表

配置全局禁用Full-mesh

$ cat << EOF | calicoctl -f -
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
  name: default
spec:
  logSeverityScreen: Info
  nodeToNodeMeshEnabled: false
  asNumber: 64512
EOF

上述命令配置完成后，再次使用命令ansible all -m shell -a '/opt/kube/bin/calicoctl node status'查看，可以看到之前所有的bgp连接都消失了。

配置 BGP node 与 Route Reflector 的连接建立规则

设定规则，通过标签区分节点角色

$ cat << EOF | calicoctl create -f -
kind: BGPPeer
apiVersion: projectcalico.org/v3
metadata:
  name: peer-to-rrs
spec:
  # 规则1：普通 bgp node 与 rr 建立连接
  nodeSelector: !has(i-am-a-route-reflector)
  peerSelector: has(i-am-a-route-reflector)

---
kind: BGPPeer
apiVersion: projectcalico.org/v3
metadata:
  name: rr-mesh
spec:
  # 规则2：route reflectors 之间也建立连接
  nodeSelector: has(i-am-a-route-reflector)
  peerSelector: has(i-am-a-route-reflector)
EOF

导出节点1和节点2的配置并修改：

calicoctl get node node1 --export -oyaml > rr01.yml
vim rr01.yaml
apiVersion: projectcalico.org/v3
kind: Node
metadata:
  creationTimestamp: null
  name: node1
  labels:
    # 增加标签，将rr标签置为true
    i-am-a-route-reflector: true
spec:
  bgp:
    ipv4Address: 192.168.2.1/24
    # 增加标签，确保同一个反射簇配置ID一致，即rr01与rr02一致，用于冗余和防环
    routeReflectorClusterID: 224.0.0.1
  orchRefs:
  - nodeName: 192.168.2.1
    orchestrator: k8s

RR1和RR2的配置同理，编写完成后应用

calicoctl apply -f rr01.yml
$ ansible all -m shell -a '/opt/kube/bin/calicoctl node status'

192.168.2.2 | SUCCESS | rc=0 >>
Calico process is running.

IPv4 BGP status
+--------------+---------------+-------+----------+-------------+
| PEER ADDRESS |   PEER TYPE   | STATE |  SINCE   |    INFO     |
+--------------+---------------+-------+----------+-------------+
| 192.168.2.1    | global        | up    | 13:29:08 | Established |
| 192.168.2.10   | node specific | up    | 13:29:10 | Established |
##省略..
| 192.168.2.9    | node specific | up    | 13:29:08 | Established |
+--------------+---------------+-------+----------+-------------+

IPv6 BGP status
No IPv6 peers found.

192.168.2.3 | SUCCESS | rc=0 >>
Calico process is running.

IPv4 BGP status
+--------------+-----------+-------+----------+-------------+
| PEER ADDRESS | PEER TYPE | STATE |  SINCE   |    INFO     |
+--------------+-----------+-------+----------+-------------+
| 192.168.2.1    | global    | up    | 13:27:01 | Established |
| 192.168.2.2    | global    | up    | 13:29:08 | Established |
+--------------+-----------+-------+----------+-------------+
##其他省略...

可以看到RR1和RR2建立连接；

其他节点分别与RR1和RR2建立连接，互相并不直连

来源：https://www.cnblogs.com/delacroix429/p/11718491.html

标签

Calico