redis+keepalived 主從熱備 實例

原創 我不是三爺 2018-09-11 02:10

    應用環境：

    操作系統：centos 5.6 x64

   redis版本：redis-2.8.10.tar.gz 

keepalived版本：keepalived-1.2.12.tar.gz



redis  master : 192.168.101.50

redis     salve : 192.168.101.51


wKiom1eQaN6ypRFMAACM7C6mGfU877.jpg-wh_50

原理：如果返回時間超過keepalived vrrp_scripts的超時時間,keepalived會進入faild 狀態,從而觸發keepalived集羣開始進行vip漂移

當 Master 掛掉，Slave 正常時, Slave接管服務，有寫權限，同時關閉主從複製功能；

當 Master 恢復正常，則從Slave同步數據，同步數據之後關閉主從複製功能，恢復Master身份，同時Slave等待Master同步數據完成之後，恢復Slave身份。


部署安裝：

1、在master  salve 上編譯安裝依賴包

      yum -y install gcc gcc+ gcc-c++ openssl openssl-devel pcre pcre-devel  


軟件包download：

               

                redis-2.8.10.tar.gz               wget http://www.915c.com/softfile/redis-2.8.10.tar.gz

                keepalived-1.2.12.tar.gz      wget http://www.keepalived.org/software/keepalived-1.2.12.tar.gz 

                gperftools-2.1.tar.gz            wget http://down.51cto.com/data/1332597

                 libunwind-1.1.tar.gz      wget http://down.51cto.com/data/1332597

2、編譯安裝 (master salve  安裝方法一致)

    

# tar xf  libunwind-1.1.tar.gz 

# cd libunwind-1.1

# CFLAGS=-fPIC ./configure 
# make CFLAGS=-fPIC 

# make CFLAGS=-fPIC install 


# tar xf gperftools-2.1.tar.gz 
# cd gperftools-2.1 

# /configure  --disable-cpu-profiler --disable-heap-profiler --disable-heap-checker --disable-debugalloc --enable-minimal 

#  make && make install 


# echo "/usr/local/lib" > /etc/ld.so.conf.d/usr_local_lib.conf 
# /sbin/ldconfig 

# cd /usr/local/lib 

# ln -sv libtcmalloc_minimal.so.4.1.2 libtcmalloc.so 

# tar xf redis-2.8.10.tar.gz 

# cd redis2.8.10 

# mkdir -p /opt/redis 

# make PREFIX=/opt/redis USE_TCMALLOC=yes FORCE_LIBC_MALLOC=yes install 



3、配置redis


創建配置文件夾以及文件 

mkdir -p /opt/redis/etc 

mkdir -p /opt/redis/run 
mkdir -p /opt/redis/data/6379 
mkdir -p /opt/redis/log 
cp /redis的解壓路徑 /redis.conf /opt/redis/etc/redis.conf 
cp /opt/redis/etc/redis.conf /opt/redis/etc/redis_6379.conf 
修改配置文件： 
vim /opt/redis/etc/redis_6379.conf 
daemonize yes 
pidfile /opt/redis/run/redis_6379.pid 
dir /opt/redis/data/6379 
logfile /opt/redis/log/redis_6379.log 
創建服務管理腳本 
vim /etc/init.d/redis 
#!/bin/sh 
PATH="/opt/redis/bin:$PATH" 
EXEC="/opt/redis/bin/redis-server" 
CLIEXEC="/opt/redis/bin/redis-cli" 
PIDFILE="/opt/redis/run/redis_6379.pid" 
CONF="/opt/redis/etc/redis_6379.conf" 
REDISPORT="6379" 

case "$1" in 
    start) 
        if [ -f $$PIDFILE ] 
        then 
                echo "$PIDFILE exists, process is already running or crashed." 
        else 
                echo "Starting Redis server..." 
                $EXEC $CONF 
        fi 
        ;; 
    stop) 
        if [ ! -f $PIDFILE ] 
        then 
                echo "$PIDFILE does not exist, process is not running." 
        else 
                PID=$(cat $PIDFILE) 
                echo "Stopping ..." 
                $CLIEXEC -p $REDISPORT shutdown 
                while [ -x /proc/${PID} ] 
                do 
                    echo "Waiting for Redis to shutdown ..." 
                    sleep 1 
                done 
                echo "Redis stopped." 
        fi 
        ;; 
    *)    
        echo "Usage: $0 {start|stop}" >&2 
        exit 1 
        ;; 
esac 

授權 chmod +x /etc/init.d/redis 


vim /etc/sysctl.conf 
在最後添加以下節點: 

vm.overcommit_memory = 1 

sysctl -p 


redis 就安裝完畢了


4、啓動測試

啓動redis 服務

# /etc/init.d/redis start

#ps -aux |grep redis  查看進程

打開防火牆6379端口


5、測試redis的主從複製


[root@master ~]# /opt/redis/bin/redis-cli -p 6379 set test1 www.baidu.com       \\通過master  set設置一個key鍵爲test1 寫入values值  www.baidu.com  

wKioL1eQdI7R_QAYAAA3avoMEWE170.jpg-wh_50

返回一個ok


在master本地測試取test1 的key值 

[root@master ~]# /opt/redis/bin/redis-cli -p 6379 get test1

wKioL1eQde6ibMZ-AABECSk-q6w612.jpg-wh_50


在salve端測試讀取master寫入的文件test1

[root@slave ~]# /opt/redis/bin/redis-cli -p 6379 get test1

wKioL1eQdyWQXY71AABQIKCxl_I611.jpg-wh_50



6、安裝keepalived 

# tar xf keepalived-1.2.12.tar.gz

# cd keepalived-1.2.12

# ./configure

# make && make install

# cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/

# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

# chmod +x /etc/init.d/keepalived

# chkconfig -add keepalived

# mkdir /etc/keepalived

# ln -s /usr/local/sbin/keepalived /usr/sbin


7、首先在master上創建如下配置

# mkdir -p /etc/keepalived

# vim /etc/keepalived/keepalived.conf

vrrp_script chk_redis { 

                script "/etc/keepalived/scripts/redis_check.sh"   ###監控腳本 

                interval 2                                        ###監控時間 

} 

vrrp_instance VI_1 { 

        state MASTER                            ###設置爲MASTER

        interface eth0                          ###監控網卡    

        virtual_router_id 51

        priority 101                            ###權重值

        authentication { 

                     auth_type PASS             ###加密 

                     auth_pass redis            ###密碼 

        } 

        track_script { 

                chk_redis                       ###執行上面定義的chk_redis

        } 

        virtual_ipaddress { 

             192.168.101.52                       ###VIP 

        }

        notify_master /etc/keepalived/scripts/redis_master.sh

        notify_backup /etc/keepalived/scripts/redis_backup.sh

        notify_fault  /etc/keepalived/scripts/redis_fault.sh

        notify_stop   /etc/keepalived/scripts/redis_stop.sh 

}

 

在Redis Master上創建redis_master腳本 


# vim /etc/keepalived/scripts/redis_master.sh

 

#!/bin/bash

REDISCLI="/opt/redis/bin/redis-cli"

LOGFILE="/var/log/keepalived-redis-state.log"

echo "[master]" >> $LOGFILE

date >> $LOGFILE

echo "Being master...." >> $LOGFILE 2>&1

echo "Run SLAVEOF cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF 192.168.101.51(slave) 6379 >> $LOGFILE  2>&1

sleep 10 #延遲10秒以後待數據同步完成後再取消同步狀態

echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1

在Redis Master上創建redis_backup腳本   

# vim /etc/keepalived/scripts/redis_backup.sh

#!/bin/bash

REDISCLI="/opt/redis/bin/redis-cli"

LOGFILE="/var/log/keepalived-redis-state.log"

echo "[backup]" >> $LOGFILE

date >> $LOGFILE

echo "Being slave...." >> $LOGFILE 2>&1

sleep 15 #延遲15秒待數據被對方同步完成之後再切換主從角色

echo "Run SLAVEOF cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF 192.168.101.51(slave)  6379 >> $LOGFILE  2>&1

 

在slave上創建如下文件：

# vim /etc/keepalived/keepalived.conf

vrrp_script chk_redis { 

                script "/etc/keepalived/scripts/redis_check.sh"   ###監控腳本 

                interval 2                                        ###監控時間 

} 

vrrp_instance VI_1 { 

        state BACKUP                                ###設置爲BACKUP 

        interface eth0                              ###監控網卡

        virtual_router_id 51 

        priority 100                                ###比MASTRE權重值低 

        authentication { 

                     auth_type PASS 

                     auth_pass redis                ###密碼與MASTRE相同

        } 

        track_script { 

                chk_redis                       ###執行上面定義的chk_redis

        } 

        virtual_ipaddress { 

             192.168.101.52                         ###VIP 

        } 

        notify_master /etc/keepalived/scripts/redis_master.sh

        notify_backup /etc/keepalived/scripts/redis_backup.sh

        notify_fault  /etc/keepalived/scripts/redis_fault.sh

        notify_stop   /etc/keepalived/scripts/redis_stop.sh 

}


在Redis salve上創建redis_master腳本 

vim /etc/keepalived/scripts/redis_master.sh

#!/bin/bash

REDISCLI="/opt/redis/bin/redis-cli"

LOGFILE="/var/log/keepalived-redis-state.log"

echo "[master]" >> $LOGFILE

date >> $LOGFILE

echo "Being master...." >> $LOGFILE 2>&1

echo "Run SLAVEOF cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF 192.168.101.50(master) 6379 >> $LOGFILE  2>&1

sleep 10 #延遲10秒以後待數據同步完成後再取消同步狀態

echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF NO ONE >> $LOGFILE 2>&1



在Redis Master上創建redis_backup腳本   

# vim /etc/keepalived/scripts/redis_backup.sh

#!/bin/bash

REDISCLI="/opt/redis/bin/redis-cli"

LOGFILE="/var/log/keepalived-redis-state.log"

echo "[backup]" >> $LOGFILE

date >> $LOGFILE

echo "Being slave...." >> $LOGFILE 2>&1

sleep 15 #延遲15秒待數據被對方同步完成之後再切換主從角色

echo "Run SLAVEOF cmd ..." >> $LOGFILE

$REDISCLI SLAVEOF 192.168.101.50  6379 >> $LOGFILE  2>&1

 

以下腳本分別在master和slave上創建

#  mkdir /etc/keepalived/scripts

#  vim /etc/keepalived/scripts/redis_check.sh

#!/bin/bash

 

ALIVE=`/opt/redis/bin/redis-cli PING`

if [ "$ALIVE" == "PONG" ]; then

  echo $ALIVE

  exit 0

else

  echo $ALIVE

  exit 1

fi

 # vim /etc/keepalived/scripts/redis_fault.sh

#!/bin/bash

LOGFILE=/var/log/keepalived-redis-state.log

echo "[fault]" >> $LOGFILE

date >> $LOGFILE

 #  vim /etc/keepalived/scripts/redis_stop.sh

#!/bin/bash

LOGFILE=/var/log/keepalived-redis-state.log

echo "[stop]" >> $LOGFILE

date >> $LOGFILE

 

授執行權  chmod +x /etc/keepalived/scripts/*.sh



8、測試高可用性。


1、啓動master和slave節點上keepalived

[root@master ~]# /etc/init.d/keepalived start

啓動 keepalived：                                          [確定]


[root@slave ~]# /etc/init.d/keepalived start

Starting keepalived:                                       [  OK  ]


查看master  的網絡地址 vip地址的綁定狀態

wKiom1eQnyCyMkVrAAErk-qEQ8E056.jpg-wh_50

查看slave

wKioL1eQn9uy6zf4AAEiexYAvds547.jpg-wh_50


9、測試vip連接redis

[root@master ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 INFO |grep -A8 Replication

wKiom1eQoaGhYo_nAAD6Ewslm_Q662.jpg-wh_50

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 INFO |grep -A8 Replication

wKioL1eQobGQaSebAAD4MZmgITY532.jpg-wh_50


10、測試插入和取出key

wKiom1eQpM_BLx-xAAEjL5npxcc573.jpg-wh_50

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 set oneday www.baidu.com

OK

You have new mail in /var/spool/mail/root

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 get oneday 

"www.baidu.com"

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.50 get oneday 

"www.baidu.com"

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.51 get oneday 

"www.baidu.com"


11、測試vip漂移和redis高可用

關閉master節點的redis服務

[root@master ~]# /etc/init.d/redis stop

Stopping ...

Redis stopped.


查看vip 地址 slave變化狀態

master

wKiom1eQqHbwsVBOAAGyCbiUOzk770.jpg-wh_50

slave

wKioL1eQqKyA6yjZAAHVCpfTypA046.jpg-wh_50


wKioL1eQqRaT3xSlAADVIxBEp4E635.jpg-wh_50


12、測試故障恢復redis master恢復

啓動master節點的redis 服務器，觀察變化情況

wKioL1eQqoTTqukZAAHJOn3UeuU591.jpg-wh_50

wKiom1eQqsvwIbGWAADHzo1zcnc332.jpg-wh_50

slave節點

wKiom1eQq2SSPZrgAAJPsGT2TWw200.jpg-wh_50

slave自動切換爲slave節點服務


到此測試完成。



備註：

以上方案存在不足之處：1、當master節點故障後，雖然slave節點接管了服務，但是畢竟slave沒有寫入的權限，導致slave需要寫入文件的時候報錯

[root@slave ~]# /opt/redis/bin/redis-cli -h 192.168.101.52 set twoday www.sina.net

(error) READONLY You can't write against a read only slave.

這個時候，如果是強制將slave節點手打或者腳本升級爲master節點，則出現當master恢復時，原master與slave已經隔離，不能完成數據同步。

2、通過哨兵Redis Sentinel 監控master slave 監聽可以實現當master出故障時，slave可以自動切換爲master並接管工作，但當master端恢復後，會一直停留在slave狀態，不會再搶佔成爲master節點。

3、拋出一個問題：假如master 是一臺服務器性能比較高，而slave是一臺性能較差的服務器，如何做到，當master節點故障時，slave自動切換爲master並且有讀寫的權限，當原master節點恢復後，自動搶佔成爲master節點？


發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章