redis cluster單機僞分佈式搭建--- 3主3從3哨兵集羣
最近公司引進微服務框架,之前的一臺redis的預存60G已經無法滿足現在的260G業務需要,經過一番考慮搭建了這套集羣 . 爲了方便我就用一臺服務器演示,生產環境中不建議這麼做(沒啥用),只爲記錄一下過程,至於精細化的配置需要在生產中自行研究
演示環境
[root@localhost ~]# cat /etc/redhat-release
CentOS Linux release 7.4.1708 (Core) redis服務部署過程
- 我下載的是redis-4.0.14.tar.gz版本的包
- 解壓安裝
tar -zxvf redis-4.0.14.tar.gz
make install - 啓動服務
./redis-server - 驗證,啓動客戶端測試
./redis-cli 127.0.0.1:6379> set ab b OK 127.0.0.1:6379> get ab "b" 127.0.0.1:6379> keys * "ab" "foo" - 關閉redis服務
./redis-cli shutdown - 設置redis後臺啓動,在/usr/local/bin目錄下
cp /usr/local/redis/redis-4.0.8/redis.conf ./
修改redis.conf這個配置文件，找到 daemonize no 這一行，將 no 改成 yes 保存
然後啓動redis時攜帶配置文件,./redis-server redis.conf - 在/usr/local/bin目錄下修改好redis.conf配置（主要是設置後臺啓動、bind、密碼、保護模式等）後
mkdir /etc/redis
cp /usr/local/bin/redis.conf /etc/redis/6379.conf - 將redis自帶的啓動腳本複製到/etc/init.d目錄下，命名爲redisd（通常都以d結尾表示是後臺自啓動服務）
cp /usr/local/redis/redis-4.0.8/utils/redis_init_script /etc/init.d/redisd - 設置爲開機自啓動 chkconfig redisd on
如果報錯redisd 不支持 chkconfig,在/etc/init.d/redisd中增加如下片段，即可再設置開機自啓動#!/bin/sh #chkconfig: 2345 90 10 #description: Redis is a persistent key-value database - chkconfig redisd on 成功之後即可以使用啓動腳本來啓動redis
僞分佈式集羣的搭建
搭建一個最小的集羣需要6臺機器（由節點投票機制決定至少需要3臺主節點，另外爲了高可用每個主節點至少要備一個從節點，所以總共需要6個節點。這裏6個節點全部設在一臺機上，通過端口號來分別。實現僞分佈式。）
- 在/usr/local/目錄下創建文件夾redisCluster，並在redisCluster目錄下創建6個文件夾
mkdir /usr/local/redisCluster
mkdir /usr/local/redisCluster/redis0{1,2,3,4,5,6} - 複製單機單實例下可正常啓動redis情況下的 /usr/local/bin 目錄下的所有文件分別到上面創建的6個文件夾中
cp /usr/local/bin/* /usr/local/redisCluster/redis01/ cp /usr/local/bin/* /usr/local/redisCluster/redis02/ cp /usr/local/bin/* /usr/local/redisCluster/redis03/ cp /usr/local/bin/* /usr/local/redisCluster/redis04/ cp /usr/local/bin/* /usr/local/redisCluster/redis05/ cp /usr/local/bin/* /usr/local/redisCluster/redis06/ - 再對每個實例裏面的redis.conf進行修改（這裏修改其中一份，然後其他5份進行復制之後修改下端口號即可)
vi /usr/local/redisCluster/redis01/redis.conf 第1處： port 6379 ————> 值 改成 7001 第2處： daemonize no ————> 值 改成 yes 第3處： pidfile /var/run/redis_6379.pid ————> 值 改成 ./redis_7001.pid 第4處： logfile "" ————> 值 改成 "./logs/redis_7001.log" 第5處： bind 127.0.0.1 ————> 值 改成 本機IP（如我這裏是192.168.25.129） 第6處： protected-mode yes ————> 值 改成 no 第7處： # cluster-enabled yes ————> 去掉註釋 第8處： # cluster-config-file nodes-6379.conf ————> 去掉註釋,並把數字改成對應端口號，如這裏是7001 第9處： # cluster-node-timeout 15000 ————> 去掉註釋 第10處： appendonly no ————> 值 改成 yes - 將上面編輯號保存的/usr/local/redisCluster/redis01/目錄下的redis.conf文件重命名爲7001.conf，然後複製5份分別放到redis02、redis03、redis04、redis05、redis06文件夾下面（可刪除掉原文件夾中的redis.conf文件），然後再對每個文件夾下的.conf文件重命名爲對應端口號.conf，接着編輯裏面的內容，將之前修改爲7001的全部修改成對應端口號保存即可
- 將ruby腳本拷貝到redisCluster目錄下
cp /usr/local/redis/redis-4.0.8/src/redis-trib.rb /usr/local/redisCluster/ - 創建redis集羣啓動、停止的腳本以及ruby創建集羣的腳本並授權
touch /usr/local/redisCluster/startCluster.sh touch /usr/local/redisCluster/shutdownCluster.sh touch /usr/local/redisCluster/createCluster.sh chmod 777 /usr/local/redisCluster/*.shstartcluster.sh 內容如下:
/usr/local/rediscluster/redis01/redis-server /usr/local/rediscluster/redis01/7001.conf /usr/local/rediscluster/redis02/redis-server /usr/local/rediscluster/redis02/7002.conf /usr/local/rediscluster/redis03/redis-server /usr/local/rediscluster/redis03/7003.conf /usr/local/rediscluster/redis04/redis-server /usr/local/rediscluster/redis04/7004.conf /usr/local/rediscluster/redis05/redis-server /usr/local/rediscluster/redis05/7005.conf /usr/local/rediscluster/redis06/redis-server /usr/local/rediscluster/redis06/7006.confshutdowncluster.sh 內容如下:
/usr/local/rediscluster/redis01/redis-cli -c -h 10.11.1.103 -p 7001 shutdown /usr/local/rediscluster/redis02/redis-cli -c -h 10.11.1.103 -p 7002 shutdown /usr/local/rediscluster/redis03/redis-cli -c -h 10.11.1.103 -p 7003 shutdown /usr/local/rediscluster/redis04/redis-cli -c -h 10.11.1.103 -p 7004 shutdown /usr/local/rediscluster/redis05/redis-cli -c -h 10.11.1.103 -p 7005 shutdown /usr/local/rediscluster/redis06/redis-cli -c -h 10.11.1.103 -p 7006 shutdowncreatecluster.sh 內容如下:
ruby redis-trib.rb create --replicas 1 10.11.1.103:7001 10.11.1.103:7002 10.11.1.103:7003 10.11.1.103:7004 10.11.1.103:7005 10.11.1.103:7006 - 用剛剛創建的腳本啓動6個實例，再用ruby腳本創建集羣
cd /usr/local/redisCluster/ ./startCluster.sh ./createcluster.sh(輸入yes) - 驗證,查看進程
[root@localhost rediscluster]# ps -ef | grep redis|grep -v grep root 23716 1 0 Aug08 ? 00:10:55 /usr/local/rediscluster/redis01/redis-server *:7001 [cluster] root 23721 1 0 Aug08 ? 00:10:53 /usr/local/rediscluster/redis02/redis-server *:7002 [cluster] root 23723 1 0 Aug08 ? 00:10:51 /usr/local/rediscluster/redis03/redis-server *:7003 [cluster] root 23731 1 0 Aug08 ? 00:13:37 /usr/local/rediscluster/redis04/redis-server *:7004 [cluster] root 23736 1 0 Aug08 ? 00:10:55 /usr/local/rediscluster/redis05/redis-server *:7005 [cluster] root 23741 1 0 Aug08 ? 00:08:59 /usr/local/rediscluster/redis06/redis-server *:7006 [cluster] - 集羣就搭建成功了。登錄任意一個節點客戶端驗證(-c 表示以集羣方式登錄)，也可以用redis客戶端工具RedisDesktopManager來可視化地管理redis數據庫。
cd /usr/local/redisCluster/redis01/ ./redis-cli -h 10.11.1.103 -p 7001 -c 10.11.1.103:7001> set aaaa b OK 10.11.1.103:7002> get aaaa -> Redirected to slot [1953] located at 10.11.1.103:7001 "b" 10.11.1.103:7001> cluster info cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:6 cluster_size:3 cluster_current_epoch:6 cluster_my_epoch:1 cluster_stats_messages_ping_sent:367555 cluster_stats_messages_pong_sent:342015 cluster_stats_messages_publish_sent:903835 cluster_stats_messages_sent:1613405 cluster_stats_messages_ping_received:342015 cluster_stats_messages_pong_received:367550 cluster_stats_messages_publish_received:1084495 cluster_stats_messages_received:1794060 10.11.1.103:7001> - 在redis01 02 03 裏配置哨兵,這裏只配置了redis01的剩餘的按照這個自行配置即可,需要注意的是監視的名稱不可重複(已標出)
sentinel.conf 配置如下:protected-mode no port 27001 sentinel deny-scripts-reconfig yes sentinel monitor **mymaster1** 10.11.1.103 7001 2 sentinel down-after-milliseconds **mymaster1** 10000 logfile "/var/log/sentinel1.log" dir "/usr/local/rediscluster/redis01" sentinel failover-timeout **mymaster1** 15000 sentinel config-epoch **mymaster1** 0 sentinel leader-epoch **mymaster1** 0 sentinel known-slave **mymaster1 10.11.1.103 7004** sentinel current-epoch 0 - 在rediscluster目錄下面創建哨兵啓動和關閉的腳本
startsentinel.sh 的內容如下/usr/local/rediscluster/redis01/redis-sentinel /usr/local/rediscluster/redis01/sentinel.conf & /usr/local/rediscluster/redis02/redis-sentinel /usr/local/rediscluster/redis02/sentinel.conf & /usr/local/rediscluster/redis03/redis-sentinel /usr/local/rediscluster/redis03/sentinel.conf &stopsentinel.sh 腳本的內容如下:
kill -9 `ps -ef | grep sentinel| grep -v grep|awk '{print $2}'` - 再次查看進程
[root@localhost rediscluster]# ps -ef | grep redis|grep -v grep root 23716 1 0 Aug08 ? 00:10:55 /usr/local/rediscluster/redis01/redis-server *:7001 [cluster] root 23721 1 0 Aug08 ? 00:10:53 /usr/local/rediscluster/redis02/redis-server *:7002 [cluster] root 23723 1 0 Aug08 ? 00:10:51 /usr/local/rediscluster/redis03/redis-server *:7003 [cluster] root 23731 1 0 Aug08 ? 00:13:37 /usr/local/rediscluster/redis04/redis-server *:7004 [cluster] root 23736 1 0 Aug08 ? 00:10:55 /usr/local/rediscluster/redis05/redis-server *:7005 [cluster] root 23741 1 0 Aug08 ? 00:08:59 /usr/local/rediscluster/redis06/redis-server *:7006 [cluster] root 23924 1 0 Aug08 ? 00:09:55 /usr/local/rediscluster/redis01/redis-sentinel *:27001 [sentinel] root 23925 1 0 Aug08 ? 00:09:38 /usr/local/rediscluster/redis02/redis-sentinel *:27002 [sentinel] root 23926 1 0 Aug08 ? 00:10:22 /usr/local/rediscluster/redis03/redis-sentinel *:27003 [sentinel] - 這樣就完成搭建,以下是需要聯網安裝的軟件
yum -y install gcc ## make install 命令需要此環境 yum -y intall ruby ## 安裝ruby（此時安裝的可能使低版本的） curl -L get.rvm.io | bash -s stable ## 安裝rvm source /usr/local/rvm/scripts/rvm ## 使安裝的rvm立即生效 rvm install 2.4.1 ## 通過rvm來安裝ruby高級版本 gem install redis ## 通過ruby的工具包gem來安裝redis接口 - 需要注意的是，在關閉redis集羣之後，再次重複步驟啓動集羣時會報錯，此時只需要刪除redis各個實例中生成的文件即可再次按步驟啓動集羣