nfs主主模式

倖福魔咒の 提交于 2020-01-11 21:26:05

1.解决了nfs单点故障;

2.使用nfs+keepalived做了主备;

  1.当nfs服务器宕机时~主备切换,依据监控keepalived主上的nfs挂了,就停掉主keepalived;依靠keepalived的nfs_check.sh监控(脚本:nfs_check.sh)

  nfs_check.sh:

    1.监控keepalived是否为主,为主:检查挂载是可用

    2.检查此机器上的nfs是否正常,不正常,关闭keepalived

  2.通过keepalived的notify_master、notify_backup、notify_fault、notify_stop的4个状态,分别对应的脚本来决定数据的同步方向

3.通过rsync+inotify实现主备间的共享目录进行同步。

VIP:192.168.109.136

master-nfs:192.168.109.137

backup-nfs:192.168.109.138

client:192.168.109.139

nfs与所有挂载机做免密安全认证,为后面判断挂载读写正常检测提供(略)

ssh-keygen

ssh-copy-id IP

一、master-nfs、backup-nfs 安装nfs服务;client安装客户端(略):参考:https://www.cnblogs.com/NGames/p/12088954.html

二、master-nfs、backup-nfs在双nfs主机上安装keepalived

yum -y install keepalived

修改配置文件:2台一样(BACKUP),主主

vim keepalived.conf

! Configuration File for keepalived
global_defs {
notification_email {
acassen@firewall.loc
failover@firewall.loc
sysadmin@firewall.loc
router_id nfs
}
notification_email_from Alexandre.Cassen@firewall.loc
smtp_server 192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
vrrp_skip_check_adv_addr
vrrp_garp_interval 0
vrrp_gna_interval 0
}

vrrp_script chk_nfs {
script "/etc/keepalived/nfs_check.sh >> /etc/keepalived/logs/check.log"
interval 3
fall 3
rise 3
}
vrrp_instance VI_1 {
state BACKUP
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
track_script {
chk_nfs
}
virtual_ipaddress {
192.168.109.136
}
notify_master /etc/keepalived/nfs_master.sh
notify_backup /etc/keepalived/nfs_backup.sh
notify_fault /etc/keepalived/nfs_fault.sh
notify_stop /etc/keepalived/nfs_stop.sh
}

 

 

 mkdir -pv /etc/keepalived/logs

---------------------------------------------------------------

vim nfs_backup.sh

vip=192.168.109.136

NUM=`ip a | grep $vip | wc -l`

LOGFILE="/etc/keepalived/logs/keepalived-nfs-state.log"

echo "[slave]" >> $LOGFILE

date >> $LOGFILE

echo "Being slave...." >> $LOGFILE 2>&1

if [ $NUM -eq 1 ]

then

echo "$vip from nfs_backup.sh 开始启用自动同步脚本" >> $LOGFILE 2>&1

nohup bash /scripts/inotifi.sh &

else

echo "无VIP from nfs_backup.sh 停止自动同步脚本" >> $LOGFILE 2>&1

kill -9 `ps aux | grep inotifi.sh | grep -v grep | awk '{print $2}'`

kill -9 `ps aux | grep inotifywait | grep -v grep | awk '{print $2}'`

fi

 

 

vim nfs_check.sh

VIP=192.168.109.136

ip=`/sbin/ip a|grep "ens33"|grep "$VIP"`

share_dir=/apprun/nfsdata

nfs_ceshi_file=/apprun/nfsdata/a.txt

if [ "$ip" != "" ];then

for i in `cat  /etc/keepalived/web_IP`;

do echo ${i};

TXT=`/usr/bin/ssh ${i} "cat $nfs_ceshi_file"`

if [ "$TXT" != "NFS server OK..." ];then

/usr/bin/ssh ${i} "umount -l $share_dir && mount -a"

fi

done

fi

port=`netstat -ntlp | grep 2049`

if [ "$port" == "" ];then

systemctl stop keepalived

fi

 

 

 echo "NFS server OK..." > /apprun/nfsdata/a.txt

 

vim nfs_fault.sh

vip=192.168.109.136

NUM=`ip a | grep $vip | wc -l`

LOGFILE="/etc/keepalived/logs/keepalived-nfs-state.log"

echo "[slave]" >> $LOGFILE

date >> $LOGFILE

echo "Being slave...." >> $LOGFILE 2>&1

if [ $NUM -eq 1 ]

then

echo "$vip from nfs_fault.sh 开始启用自动同步脚本" >> $LOGFILE 2>&1

nohup bash /scripts/inotifi.sh &

else

echo "无VIP from nfs_fault.sh 停止自动同步脚本" >> $LOGFILE 2>&1

kill -9 `ps aux | grep inotifi.sh | grep -v grep | awk '{print $2}'`

kill -9 `ps aux | grep inotifywait | grep -v grep | awk '{print $2}'`

fi

 

 

 

 

 

 

vim nfs_master.sh

vip=192.168.109.136

NUM=`ip a | grep $vip | wc -l`

LOGFILE="/etc/keepalived/logs/keepalived-nfs-state.log"

echo "[master]" >> $LOGFILE

date >> $LOGFILE

echo "Being master...." >> $LOGFILE 2>&1

sleep 1

if [ $NUM -eq 1 ]

then

echo "$vip from nfs_master.sh 开始启用自动同步脚本" >> $LOGFILE 2>&1

nohup bash /scripts/inotifi.sh &

else

echo "无VIP from nfs_master.sh 停止自动同步脚本" >> $LOGFILE 2>&1

kill -9 `ps aux | grep inotifi.sh | grep -v grep | awk '{print $2}'`

kill -9 `ps aux | grep inotifywait | grep -v grep | awk '{print $2}'`

fi

 

 

vim nfs_stop.sh

vip=192.168.109.136

NUM=`ip a | grep $vip | wc -l`

LOGFILE="/etc/keepalived/logs/keepalived-nfs-state.log"

echo "[slave]" >> $LOGFILE

date >> $LOGFILE

echo "Being slave...." >> $LOGFILE 2>&1

if [ $NUM -eq 1 ]

then

echo "$vip from nfs_stop.sh 开始启用自动同步脚本" >> $LOGFILE 2>&1

nohup bash /scripts/inotifi.sh &

else

echo "无VIP from nfs_stop.sh 停止自动同步脚本" >> $LOGFILE 2>&1

kill -9 `ps aux | grep inotifi.sh | grep -v grep | awk '{print $2}'`

kill -9 `ps aux | grep inotifywait | grep -v grep | awk '{print $2}'`

fi

 

 

web_IP 记录需要挂载的机器ip

 

 -------------------------------------------------------------------------------------------

配置文件同步:

rsync -avz --progress -e ssh /etc/keepalived/* 192.168.109.138:/etc/keepalived/

 

1.此时启动keepalived会因没有同步数据脚本/scripts/inotifi.sh ,可以先配置下面的数据同步

2.要先nfs,不然检测会关闭keepalived

systemctl restart keepalived.service

systemctl enable keepalived.service

 

 

=====================================================================================================

 三、创建共享目录

mkdir -pv /apprun/nfsdata

 

四、共享目录资源同步

1.情况一:master_nfs为(192.168.109.137);backup_nfs为(192.168.109.138)

安装:backup_nfs为(192.168.109.138)

\\安装rsync

yum -y install rsync

vim /etc/rsyncd.conf

pidfile = /var/run/rsyncd.pid

lock file = /var/run/rsyncd.lock

log file = /var/log/rsyncd.log

[backup]

path = /apprun/nfsdata

comment = client(192.168.109.138)

uid = root

gid = root

port = 873

ignore errors

use chroot = no

read only = no

list = no

max connections = 200

timeout = 600

auth users = wangbocheng

secrets file = /etc/rsync.pass

 

 

 \\创建用户认证文件

echo "wangbocheng:wang123" > /etc/rsync.pass

 

  chmod 600 /etc/rsync.pass

\\启动

systemctl restart rsyncd

systemctl enable rsyncd

---------------------------------------------------

master_nfs为(192.168.109.137)

yum -y install rsync inotify-tools

依据系统版本选择源:

wget http://mirrors.aliyun.com/repo/epel-7.repo

yum -y install inotify-tools

echo "wang123" > /etc/rsync.pass

 

chmod 600 /etc/rsync.pass

手动同步命令:(手动数据同步)

rsync -avzp --port=873 --progress --delete /apprun/nfsdata/ wangbocheng@192.168.109.138::backup --password-file=/etc/rsync.pass

自动同步

\\编写脚本:vim /scripts/inotifi.sh

host=192.168.109.138
src=/apprun/nfsdata/
des=backup
password=/etc/rsync.pass
user=wangbocheng
inotifywait=/usr/bin/inotifywait
$inotifywait -mrq --timefmt '%Y%m%d %H:%M' --format '%T %w%f%e' -e modify,delete,create,attrib $src | while read files ;do
rsync -avzP --delete --timeout=100 --password-file=${password} $src $user@$host::$des
echo "${files} was rsynced" >>/tmp/rsync.log 2>&1

done 

 

 //启动脚本

nohup bash /scripts/inotifi.sh &

\\查看是否同步

在master-nfs共享目录下写入文件,查看日志,查看 backup-nfs目录同步情况

######设置脚本的开机自启动 (依各人环境需求是否配置,此处无需)

######touch /etc/rc.d/rc.local

######chmod +x /etc/rc.d/rc.local

######echo "nohup /bin/bash /scripts/inotify.sh" >> /etc/rc.d/rc.local

######重启后查看

 

2.情况二:master_nfs为(192.168.109.138);backup_nfs为(192.168.109.137)

安装:backup_nfs为(192.168.109.137)

\\安装rsync

yum -y install rsync

vim /etc/rsyncd.conf

pidfile = /var/run/rsyncd.pid

lock file = /var/run/rsyncd.lock

log file = /var/log/rsyncd.log

[backup]

path = /apprun/nfsdata

comment = client(M138)

uid = root

gid = root

port = 873

ignore errors

use chroot = no

read only = no

list = no

max connections = 200

timeout = 600

auth users = wangbocheng2

secrets file = /etc/rsync2.pass

 

 

  \\创建用户认证文件

echo "wangbocheng2:wang123" > /etc/rsync2.pass

 

  chmod 600 /etc/rsync2.pass

\\启动

systemctl restart rsyncd

systemctl enable rsyncd

---------------------------------------------------

master_nfs为(192.168.109.138)

yum -y install rsync inotify-tools

依据系统版本选择源:

wget http://mirrors.aliyun.com/repo/epel-7.repo

yum -y install inotify-tools

echo "wang123" > /etc/rsync2.pass

 

 chmod 600 /etc/rsync2.pass

手动同步命令:(手动数据同步)

rsync -avzp --port=873 --progress --delete /apprun/nfsdata/ wangbocheng2@192.168.109.137::backup --password-file=/etc/rsync2.pass

自动同步

\\编写脚本:vim /scripts/inotifi.sh

host=192.168.109.137
src=/apprun/nfsdata/
des=backup
password=/etc/rsync2.pass
user=wangbocheng2
inotifywait=/usr/bin/inotifywait
$inotifywait -mrq --timefmt '%Y%m%d %H:%M' --format '%T %w%f%e' -e modify,delete,create,attrib $src | while read files ;do
rsync -avzP --delete --timeout=100 --password-file=${password} $src $user@$host::$des
echo "${files} was rsynced" >>/tmp/rsync.log 2>&1

done 

 

 

 //启动脚本(1.2情况只能是一边启动同步脚本,不然数据混乱)

nohup bash /scripts/inotifi.sh &

\\查看是否同步

在master-nfs共享目录下写入文件,查看日志,查看 backup-nfs目录同步情况

 

 ===============================================================================

nfs服务安装(略)参考:https://www.cnblogs.com/NGames/p/12088954.html

 

日志:

keepalived 状态切换日志相关记录:tail -f /etc/keepalived/logs/keepalived-nfs-state.log

健康检查相关日志记录:tail -f /etc/keepalived/logs/check.log

数据源同步相关日志记录:tail -f /tmp/rsync.log

 

 

 

==============================================

已验证,

情况1:nfs意外关闭;状态切换正常,数据同步正常

情况2:keepalived意外关闭;状态切换正常,数据同步正常

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!