keepalived+nginx高可用

背景

通常情況下,利用nginx進行負載均衡,使後端服務高可用,某個後臺服務出現問題可以動態剔除,不影響系統的正常運行,但是nginx只有一個,這時候是個瓶頸,會出現單點故障,nginx出現問題,服務將會出現中斷,影響系統的正常運行。

 

keepalived解決nginx單點問題

利用keepalived故障切換轉移的功能,部署多個nginx服務,比如兩個,一個master對外服務,另外一個處於backup熱備服務,對外通過vip進行訪問。當master服務出現問題的時候,backup熱備服務的機器將會動態接管vip,接管vip後backup熱備服務將會對外服務,從而解決nginx單點故障問題。

keepalived配置

[root@k8snode01 keepalived]# more keepalived.conf 
! Configuration File for keepalived

global_defs {
   notification_email {
     [email protected]
     [email protected]
     [email protected]
   }
   notification_email_from [email protected]
   smtp_server 192.168.200.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL_01
}

vrrp_script chk_http_port {
   script "/etc/keepalived/check_ng.sh"
   interval 2
   weight -20
}

vrrp_instance VI_1 {
    state MASTER  #備機爲BACKUP
    interface ens33
    virtual_router_id 100
    priority 100 #備機爲50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }

notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"

track_script {
   chk_http_port
   }

virtual_ipaddress {
        192.168.23.200
    }
}

[root@k8snode01 keepalived]# 
[root@k8snode01 keepalived]# more check_ng.sh  #nginx檢查腳本
#!/bin/bash
currdate=`date '+%Y-%m-%d %H:%M:%S'`
count=`ps -aux | grep -v grep | grep 'nginx.conf' | wc -l`
echo "${currdate}---->ok:${count}" >>/etc/keepalived/checkng.log
result=$(echo "000${count}" | grep "0000")
if [[ "$result" != "" ]]; then
    echo "${currdate}---->/etc/init.d/keepalived stop" >>/etc/keepalived/checkng.log
    /etc/init.d/keepalived stop
    exit 1
else
    exit 0
fi
[root@k8snode01 keepalived]# more notify.sh  #切換主備通知腳本
#!/bin/bash
currdate=`date '+%Y-%m-%d %H:%M:%S'`

echo "${currdate}---->${1}" >>/etc/keepalived/notify.log
[root@k8snode01 keepalived]# 

 

故障演練

1)查看vip,vip在192.168.23.101機器上
[root@k8snode01 keepalived]# ip addr  #vip在192.168.23.101機器
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:a3:b8:9f brd ff:ff:ff:ff:ff:ff
    inet 192.168.23.101/24 brd 192.168.23.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.23.200/32 scope global ens33
       valid_lft forever preferred_lft forever
[root@k8snode01 keepalived]#

[root@k8snode02 ~]# ip addr #192.168.23.102機器無vip
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:32:05:d5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.23.102/24 brd 192.168.23.255 scope global ens33
       valid_lft forever preferred_lft forever
[root@k8snode02 ~]#

2)使用實ip訪問nginx,爲了演示方便,192.168.23.101的nginx返回192.168.23.101,192.168.23.102的nginx返回192.168.23.102
[root@k8snode02 ~]# curl "192.168.23.101:8888"  #返回nginx所在機器的ip
192.168.23.101

[root@k8snode02 ~]# curl "192.168.23.102:8888" #返回nginx所在機器的ip
192.168.23.102

[root@k8snode02 ~]#

3)使用vip訪問nginx,正常應該返回vip所在機器的實ip,此時返回192.168.23.101
[root@k8snode02 ~]# curl "192.168.23.200:8888" #返回vip所在機器的ip
192.168.23.101

[root@k8snode02 ~]# curl "192.168.23.200:8888"  #返回vip所在機器的ip
192.168.23.101

[root@k8snode02 ~]# curl "192.168.23.200:8888"  #返回vip所在機器的ip
192.168.23.101

[root@k8snode02 ~]# 

4)模擬主的nginx故障,可以把192.168.23.101機器關閉或停止nginx服務,使vip漂移到192.168.23.102,讓備nginx接管服務
[root@k8snode01 keepalived]# ps -ef|grep nginx  #查看nginx進程
root     23081     1  0 08:11 ?        00:00:00 nginx: master process /data/nginxinstall/sbin/nginx -c /data/nginxinstall/conf/nginx.conf
nobody   23082 23081  0 08:11 ?        00:00:00 nginx: worker process
root     28703 24828  0 08:30 pts/1    00:00:00 grep --color=auto nginx
[root@k8snode01 keepalived]# pkill nginx #停止nginx進程
[root@k8snode01 keepalived]# ps -ef|grep nginx #nginx進程已停止
root     28746 24828  0 08:30 pts/1    00:00:00 grep --color=auto nginx
[root@k8snode01 keepalived]# ip addr  #192.168.23.101機器的vip消失
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:a3:b8:9f brd ff:ff:ff:ff:ff:ff
    inet 192.168.23.101/24 brd 192.168.23.255 scope global ens33
       valid_lft forever preferred_lft forever
[root@k8snode01 keepalived]#

[root@k8snode02 ~]# ip addr #vip漂移到192.168.23.102
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:32:05:d5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.23.102/24 brd 192.168.23.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.23.200/32 scope global ens33
       valid_lft forever preferred_lft forever
[root@k8snode02 ~]#

[root@k8snode02 ~]# curl "192.168.23.200:8888" #被nginx192.168.23.102正常接管服務
192.168.23.102

[root@k8snode02 ~]# curl "192.168.23.200:8888" #被nginx192.168.23.102正常接管服務
192.168.23.102

[root@k8snode02 ~]# 

5)將192.168.23.101機器的nginx和keepalived服務拉起,由於192.168.23.101機器優先級高,將搶佔vip
[root@k8snode01 keepalived]# /data/nginxinstall/sbin/nginx -c /data/nginxinstall/conf/nginx.conf #啓動nginx
[root@k8snode01 keepalived]# /etc/init.d/keepalived start #啓動keepalived
Starting keepalived (via systemctl):                       [  OK  ]
[root@k8snode01 keepalived]# ip addr #vip搶佔在192.168.23.101
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:a3:b8:9f brd ff:ff:ff:ff:ff:ff
    inet 192.168.23.101/24 brd 192.168.23.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet 192.168.23.200/32 scope global ens33
       valid_lft forever preferred_lft forever
[root@k8snode01 keepalived]#

[root@k8snode02 ~]# ip addr #192.168.23.102上的vip消失
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:0c:29:32:05:d5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.23.102/24 brd 192.168.23.255 scope global ens33
       valid_lft forever preferred_lft forever
[root@k8snode02 ~]# 
[root@k8snode02 ~]# curl "192.168.23.200:8888" #主nginx搶佔服務
192.168.23.101

[root@k8snode02 ~]# curl "192.168.23.200:8888" #主nginx搶佔服務
192.168.23.101

[root@k8snode02 ~]#


 

 

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章