周五晚上LVS集群的一台LB由于CPU单核耗尽挂掉了,最后查到原因是网卡调优没有生效,今天查了一下网卡调优的资料,感谢同事higkoo给我讲解了一下这方面的东西,下面是关于我们公司网卡调优的相关知识,欢迎大家共同探讨,让我们的机器跑的更high。
1、Broadcom的网卡建议关闭GRO功能
1 2 3 4 | ethtool -K eth0 gro off ethtool -K eth1 gro off ethtool -K eth2 gro off ethtool -K eth3 gro off |
2、关闭irqbalance服务并手动分配网卡中断
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | service irqbalance stop chkconfig irqbalance off # 查看网卡中断号 grep
/proc/interrupts # 分配到每颗颗CPU核上 cat /proc/irq/ {84,85,86,87,88,89,90,91,92,93} /smp_affinity echo
/proc/irq/84/smp_affinity echo
/proc/irq/85/smp_affinity echo
/proc/irq/86/smp_affinity echo
/proc/irq/87/smp_affinity echo
/proc/irq/88/smp_affinity echo
/proc/irq/89/smp_affinity echo
/proc/irq/90/smp_affinity echo
/proc/irq/91/smp_affinity echo
/proc/irq/92/smp_affinity echo
/proc/irq/93/smp_affinity PS:smp_affinity的值可以用下面脚本算哈,此脚本来自:http: //rfyiamcool .blog.51cto.com /1030776/1335700 #!/bin/bash # echo "统计cpu的16进制" [ $ # -ne 1 ] && echo ‘$1 is Cpu core number’ && exit 1 CCN=$1 echo “Print eth0 affinity” for ((i=0; i<${CCN}; i++)) do echo ============================== echo "Cpu Core $i is affinity" ((affinity=(1<<i))) echo "obase=16;${affinity}"
bc done 使用方法:sh 脚本名字 空格 cpu核数 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | # Enable RPS (Receive Packet Steering) rfc=4096 cc=$( grep
/proc/cpuinfo ) rsfe=$( echo
bc ) sysctl -w net.core.rps_sock_flow_entries=$rsfe for
in $( ls /sys/class/net/eth * /queues/rx- * /rps_cpus ) do echo fff > $fileRps done for
in $( ls /sys/class/net/eth * /queues/rx- * /rps_flow_cnt ) do echo $rfc > $fileRfc done tail /sys/class/net/eth * /queues/rx- */{rps_cpus,rps_flow_cnt} |
献上一个完整的脚本:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | vi /opt/sbin/change_irq .sh #!/bin/bash ethtool -K eth0 gro off ethtool -K eth1 gro off ethtool -K eth2 gro off ethtool -K eth3 gro off service irqbalance stop chkconfig irqbalance off cat /proc/irq/ {84,85,86,87,88,89,90,91,92,93} /smp_affinity echo
/proc/irq/84/smp_affinity echo
/proc/irq/85/smp_affinity echo
/proc/irq/86/smp_affinity echo
/proc/irq/87/smp_affinity echo
/proc/irq/88/smp_affinity echo
/proc/irq/89/smp_affinity echo
/proc/irq/90/smp_affinity echo
/proc/irq/91/smp_affinity echo
/proc/irq/92/smp_affinity echo
/proc/irq/93/smp_affinity # Enable RPS (Receive Packet Steering) rfc=4096 cc=$( grep
/proc/cpuinfo ) rsfe=$( echo
bc ) sysctl -w net.core.rps_sock_flow_entries=$rsfe for
in $( ls /sys/class/net/eth * /queues/rx- * /rps_cpus ) do echo fff > $fileRps done for
in $( ls /sys/class/net/eth * /queues/rx- * /rps_flow_cnt ) do echo $rfc > $fileRfc done tail /sys/class/net/eth * /queues/rx- */{rps_cpus,rps_flow_cnt} chmod
/opt/sbin/change_irq .sh echo "/opt/sbin/change_irq.sh"
/etc/rc . local PS:记得修改网卡中断号,别直接拿来用哦 |