Redis數據庫主從複製

Redis數據庫主從複製

一、主從複製的概述；
二、同步詳解；
三、案例：配置主從複製；

一、主從複製的概述：

概述：爲了分擔讀寫壓力，Redis支持主從複製，Redis的主從結構可以採用一主多從或者級聯結構，Redis主從複製可以根據是否是全量分爲全量同步和增量同步。下圖爲級聯結構。

二、同步詳解：

同步類型：全量同步、增量同步
1.全量同步：
　　Redis全量複製一般發生在Slave初始化階段，這時Slave需要將Master上的所有數據都複製一份。具體步驟如下： 
　　1）從服務器連接主服務器，發送SYNC命令（從服務器向主服務器初次同步時，不會影響主服務器接收客戶端的請求）； 
　　2）主服務器接收到SYNC命名後，開始執行BGSAVE命令生成RDB文件並使用緩衝區記錄此後執行的所有寫命令； 
　　3）主服務器BGSAVE執行完後，向所有從服務器發送快照文件，並在發送期間繼續記錄被執行的寫命令； 
　　4）從服務器收到快照文件後丟棄所有舊數據，載入收到的快照； 
　　5）主服務器快照發送完畢後開始向從服務器發送緩衝區中的寫命令； 
　　6）從服務器完成對快照的載入，開始接收命令請求，並執行來自主服務器緩衝區的寫命令； 

完成上面幾個步驟後就完成了從服務器數據初始化的所有操作，從服務器此時可以接收來自用戶的讀請求。
2.增量同步：
　　Redis增量複製是指Slave初始化後開始正常工作時主服務器發生的寫操作同步到從服務器的過程。 增量複製的過程主要是主服務器每執行一個寫命令就會向從服務器發送相同的寫命令，從服務器接收並執行收到的寫命令。

同步策略：
主從剛剛連接的時候，進行全量同步；
全同步結束後，進行增量同步。當然，如果有需要，slave 在任何時候都可以發起全量同步。redis 策略是，無論如何，首先會嘗試進行增量同步，如不成功，要求從機進行全量同步。
注：
如果多個Slave斷線了，需要重啓的時候，因爲只要Slave啓動，就會發送sync請求和主機全量同步，當多個同時出現的時候，可能會導致Master IO劇增導致宕機。
建議開啓master主服務器的持久化功能，避免出現master重啓後，數據無法恢復；

三、案例：配置主從複製：

實驗步驟：
安裝並配置master角色的redis服務；
安裝並配置slave角色的redis服務（雙實例）；
配置master角色的文件；
配置slave角色的文件；
驗證主從複製；

安裝並配置master角色的redis服務；

[root@master ~]# wget http://download.redis.io/releases/redis-4.0.9.tar.gz
[root@master ~]# tar zxvf redis-4.0.9.tar.gz 
[root@master ~]# cd redis-4.0.9
[root@master  redis-4.0.9]# make 
[root@master  redis-4.0.9]# echo $?
[root@master redis-4.0.9]# cd
[root@master ~]# mkdir -p /usr/local/redis
[root@master ~]# cp /root/redis-4.0.9/src/redis-server /usr/local/redis/        ##服務端程序
[root@master ~]# cp /root/redis-4.0.9/src/redis-cli /usr/local/redis/           ##客戶端程序
[root@master ~]# cp /root/redis-4.0.9/redis.conf /usr/local/redis/              ##主配置文件
[root@master ~]# ls /usr/local/redis/
redis-cli  redis.conf  redis-server
[root@master ~]# sed -i '/^bind 127.0.0.1$/s/127.0.0.1/192.168.100.101/g' /usr/local/redis/redis.conf   
[root@master ~]# sed -i '/protected-mode/s/yes/no/g' /usr/local/redis/redis.conf                ##關閉redis的保護模式
[root@master ~]# sed -i '/daemonize/s/no/yes/g' /usr/local/redis/redis.conf                 ##開啓redis的後臺守護進程模式
[root@master ~]# sed -i '/requirepass/s/foobared/123123/g' /usr/local/redis/redis.conf      ##設置redis的密碼爲123123
[root@master ~]# sed -i '/requirepass 123123/s/^#//g' /usr/local/redis/redis.conf           ##開啓redis的密碼
[root@master ~]# ln -s /usr/local/redis/redis-cli /usr/local/bin/redis
[root@master ~]# cat <<END >>/etc/init.d/redis
#!/bin/sh
# chkconfig: 2345 80 90
# description: Start and Stop redis
#PATH=/usr/local/bin:/sbin:/usr/bin:/bin
REDISPORT=6379
EXEC=/usr/local/redis/redis-server
REDIS_CLI=/usr/local/redis/redis-cli
PIDFILE=/var/run/redis_6379.pid
CONF="/usr/local/redis/redis.conf"
AUTH="123123"
LISTEN_IP=\$(netstat -utpln |grep redis-server |awk '{print \$4}'|awk -F':' '{print \$1}')

case "\$1" in
    start)
        if [ -f \$PIDFILE ]
        then
                echo "\$PIDFILE exists, process is already running or crashed"
        else
                echo "Starting Redis server..."
                \$EXEC \$CONF
        fi
        if [ "\$?"="0" ]
        then
              echo "Redis is running..."
        fi
        ;;
    stop)
        if [ ! -f \$PIDFILE ]
        then
                echo "\$PIDFILE does not exist, process is not running"
        else
                PID=\$(cat \$PIDFILE)
                echo "Stopping ..."
                \$REDIS_CLI -h \$LISTEN_IP -p \$REDISPORT -a \$AUTH SHUTDOWN
                while [ -x \${PIDFILE} ]
               do
                    echo "Waiting for Redis to shutdown ..."
                    sleep 1
                done
                echo "Redis stopped"
        fi
        ;;
   restart|force-reload)
        \${0} stop
        \${0} start
        ;;
  *)
    echo "Usage: /etc/init.d/redis {start|stop|restart|force-reload}" >&2
        exit 1
esac
END
[root@master ~]# chmod 755 /etc/init.d/redis
[root@master ~]# chkconfig --add redis
[root@master ~]# /etc/init.d/redis start
Starting Redis server...
4390:C 04 May 02:16:45.232 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
4390:C 04 May 02:16:45.232 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=4390, just started
4390:C 04 May 02:16:45.232 # Configuration loaded
Redis is running...
[root@master ~]# netstat -utpln |grep redis
tcp        0      192.168.100.101:6379            0.0.0.0:*               LISTEN      4204/redis-server *
[root@master ~]# redis -h 192.168.100.101 -a 123123 -p 6379
192.168.100.101:6379> exit

安裝並配置slave角色的redis服務（雙實例）；
[root@slave ~]# wget http://download.redis.io/releases/redis-4.0.9.tar.gz
[root@slave ~]# tar zxvf redis-4.0.9.tar.gz 
[root@slave ~]# cd redis-4.0.9
[root@slave redis-4.0.9]# make 
[root@slave  redis-4.0.9]# echo $?
[root@slave redis-4.0.9]# cd
[root@slave ~]# mkdir -p /usr/local/redis
[root@slave ~]# cp /root/redis-4.0.9/src/redis-server /usr/local/redis/     ##服務端程序
[root@slave ~]# cp /root/redis-4.0.9/src/redis-cli /usr/local/redis/            ##客戶端程序
[root@slave ~]# cp /root/redis-4.0.9/redis.conf /usr/local/redis/               ##主配置文件
[root@slave ~]# ls /usr/local/redis/
redis-cli  redis.conf  redis-server
[root@slave ~]# sed -i '/^bind 127.0.0.1$/s/127.0.0.1/192.168.100.102/g' /usr/local/redis/redis.conf    
[root@slave ~]# sed -i '/protected-mode/s/yes/no/g' /usr/local/redis/redis.conf             ##關閉redis的保護模式
[root@slave ~]# sed -i '/daemonize/s/no/yes/g' /usr/local/redis/redis.conf                  ##開啓redis的後臺守護進程模式
[root@slave ~]# sed -i '/requirepass/s/foobared/123123/g' /usr/local/redis/redis.conf       ##設置redis的密碼爲123123
[root@slave ~]# sed -i '/requirepass 123123/s/^#//g' /usr/local/redis/redis.conf            ##開啓redis的密碼
[root@slave ~]# cp /usr/local/redis/redis.conf /usr/local/redis/redis01.conf
[root@slave ~]# sed -i '92s/6379/6380/g' /usr/local/redis/redis01.conf      ##更改監聽端口
[root@slave ~]# sed -i '158s/6379/6380/g' /usr/local/redis/redis01.conf     ##更改PID文件
[root@slave ~]# sed -i '171s/^\(.\).\{9\}/logfile "\/usr\/local\/redis\/redis01.log"/g' /usr/local/redis/redis01.conf               ##更改日誌文件位置，9代表舊內容的字節數
[root@slave ~]# ln -s /usr/local/redis/redis-cli /usr/local/bin/redis
[root@slave ~]# cat <<END >>/etc/init.d/redis
#!/bin/sh
# chkconfig: 2345 80 90
# description: Start and Stop redis
#PATH=/usr/local/bin:/sbin:/usr/bin:/bin
REDISPORT=6379
EXEC=/usr/local/redis/redis-server
REDIS_CLI=/usr/local/redis/redis-cli
PIDFILE=/var/run/redis_6379.pid
CONF="/usr/local/redis/redis.conf"
AUTH="123123"
LISTEN_IP=\$(netstat -utpln |grep redis-server |awk '{print \$4}'|awk -F':' '{print \$1}' |uniq)

case "\$1" in
    start)
        if [ -f \$PIDFILE ]
        then
                echo "\$PIDFILE exists, process is already running or crashed"
        else
                echo "Starting Redis server..."
                \$EXEC \$CONF
        fi
        if [ "\$?"="0" ]
        then
              echo "Redis is running..."
        fi
        ;;
    stop)
        if [ ! -f \$PIDFILE ]
        then
                echo "\$PIDFILE does not exist, process is not running"
        else
                PID=\$(cat \$PIDFILE)
                echo "Stopping ..."
                \$REDIS_CLI -h \$LISTEN_IP -p \$REDISPORT -a \$AUTH SHUTDOWN
                while [ -x \${PIDFILE} ]
               do
                    echo "Waiting for Redis to shutdown ..."
                    sleep 1
                done
                echo "Redis stopped"
        fi
        ;;
   restart|force-reload)
        \${0} stop
        \${0} start
        ;;
  *)
    echo "Usage: /etc/init.d/redis {start|stop|restart|force-reload}" >&2
        exit 1
esac
END
[root@slave ~]# chmod 755 /etc/init.d/redis
[root@slave ~]# chkconfig --add redis
[root@slave ~]# cp /etc/init.d/redis /etc/init.d/redis01
[root@slave ~]# sed -i 's/6379/6380/g' /etc/init.d/redis01
[root@slave ~]# sed -i '/CONF=/s/redis.conf/redis01.conf/g' /etc/init.d/redis01
[root@slave ~]# /etc/init.d/redis start
Starting Redis server...
4390:C 04 May 02:16:45.232 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
4390:C 04 May 02:16:45.232 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=4390, just started
4390:C 04 May 02:16:45.232 # Configuration loaded
Redis is running...
[root@slave ~]# /etc/init.d/redis01 start
Starting Redis server...
4390:C 04 May 02:16:45.232 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
4390:C 04 May 02:16:45.232 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=4390, just started
4390:C 04 May 02:16:45.232 # Configuration loaded
Redis is running...
[root@slave ~]# netstat -utpln |grep redis
tcp        0      0 192.168.100.102:6379    0.0.0.0:*               LISTEN      11864/redis-server  
tcp        0      0 192.168.100.102:6380    0.0.0.0:*               LISTEN      11877/redis-server
[root@ slave ~]# redis -h 192.168.100.102 -a 123123 -p 6379
192.168.100.102:6379> exit
[root@ slave ~]# redis -h 192.168.100.102 -a 123123 -p 6380
192.168.100.102:6380> exit

配置master角色的文件；
[root@master ~]# sed -i '450s/^\(.\).\{22\}/min-slaves-to-write 2/g' /usr/local/redis/redis.conf
##設置slave節點的數量，如果slave節點數量少於此值，那麼master節點將停止客戶端的一切寫請求
[root@master ~]# sed -n '451s/^\(.\).\{22\}/min-slaves-max-lag 10/g' /usr/local/redis/redis.conf
##master與slave之間同步數據的超時時間，若超過此時間，master節點將停止客戶端的一切寫操作
[root@master ~]# /etc/init.d/redis restart
Stopping ...
Redis stopped
Starting Redis server...
1638:C 15 May 16:32:08.301 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1638:C 15 May 16:32:08.301 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=1638, just started
1638:C 15 May 16:32:08.301 # Configuration loaded
Redis is running...

配置slave角色的文件；
[root@slave ~]# sed -i '281s/^\(.\).\{32\}/slaveof 192.168.100.101 6379/g' /usr/local/redis/redis.conf
##指定master的ip地址以及端口
[root@slave ~]# sed -i '288s/^\(.\).\{29\}/masterauth 123123/g' /usr/local/redis/redis.conf
##指定master的連接密碼
[root@slave ~]# /etc/init.d/redis restart
/var/run/redis_6379.pid does not exist, process is not running
Starting Redis server...
4387:C 18 May 03:24:00.027 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
4387:C 18 May 03:24:00.027 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=4387, just started
4387:C 18 May 03:24:00.027 # Configuration loaded
Redis is running...
[root@slave ~]# sed -i '281s/^\(.\).\{32\}/slaveof 192.168.100.101 6379/g' /usr/local/redis/redis01.conf
[root@slave ~]# sed -i '288s/^\(.\).\{29\}/masterauth 123123/g' /usr/local/redis/redis01.conf
[root@slave ~]# /etc/init.d/redis01 restart
Stopping ...
Redis stopped
Starting Redis server...
Redis is running...

驗證主從複製；
[root@master ~]# redis -h 192.168.100.101 -a 123123 -p 6379         ##在master節點上創建鍵值對
192.168.100.101:6379> set name xiaoming
OK
192.168.100.101:6379> keys *
1) "name"
192.168.100.101:6379> get name
"xiaoming"
192.168.100.101:6379> info replication                      ##查看複製信息
# Replication
role:master
connected_slaves:2
min_slaves_good_slaves:2
slave0:ip=192.168.100.102,port=6380,state=online,offset=522,lag=1
slave1:ip=192.168.100.102,port=6379,state=online,offset=522,lag=1
master_replid:46ec2c7e971d337d060d183d3a0c1313c5dd1683
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:522
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:522
192.168.100.101:6379> exit

[root@slave ~]# redis -h 192.168.100.102 -p 6379 -a 123123          ##登錄slave節點驗證鍵值同步情況，並測試無法寫入
192.168.100.102:6379> keys *
1) "name"
192.168.100.102:6379> get name
"xiaoming"
192.168.100.102:6380> set name1 xiaohong
(error) READONLY You can't write against a read only slave.
192.168.100.102:6379> exit
[root@slave ~]# redis -h 192.168.100.102 -p 6380 -a 123123
192.168.100.102:6380> keys *
1) "name"
192.168.100.102:6380> get name
"xiaoming"
192.168.100.102:6380> exit

[root@master ~]# reboot                         ##重啓master節點，驗證redis默認的RDB持久化
[root@master ~]# /etc/init.d/redis restart
[root@master ~]# redis -h 192.168.100.101 -a 123123 -p 6379
192.168.100.101:6379> keys *
1) "name"
192.168.100.101:6379> exit

[root@slave ~]# redis -h 192.168.100.102 -p 6380 -a 123123 
192.168.100.102:6380> keys *
1) "name"
192.168.100.102:6380> get name
"xiaoming"
192.168.100.102:6380> exit