前言:本文主要講解keepalived+haproxy,等試驗完成,後面會附上keepalived+nginx的思路,原理幾乎相同,相信能看懂keepalived+haproxy的朋友,亦能很簡單的看懂keepalived+nginx
拓撲:
準備工作:
1).高可用集羣節點基於名稱互相訪問(兩節點都需配置,略)
# vim /etc/sysconfig/network
# vim /etc/hosts
2).高可用集羣節點ssh互信(兩節點都需配置)
a.com:
# ssh-keygen -t rsa -P ''
# ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]
b.com:
# ssh-keygen -t rsa -P ''
# ssh-copy-id -i /root/.ssh/id_rsa.pub [email protected]
3).高可用集羣節點時間同步,推薦使用ntpdate向時間服務器同步
爲了簡單實驗,所以採用date命令同步時間
# date 00:00:00 ; ssh [email protected] 'date 00:00:00'
安裝軟件:
# yum install -y keepalived haproxy
配置:
1).修改haproxy配置文件:(a.com節點上操作)
# vim/etc/haproxy/haproxy.cfg
#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
# to have thesemessages end up in /var/log/haproxy.log you will
# need to:
#
# 1) configure syslogto accept network log events. This isdone
# by adding the '-r' option to theSYSLOGD_OPTIONS in
# /etc/sysconfig/syslog
#
# 2) configure local2events to go to the /var/log/haproxy.log
# file. A line like the following can be addedto
# /etc/sysconfig/syslog
#
# local2.* /var/log/haproxy.log
#
log 127.0.0.1 local2 #日誌通過rsyslog保存
chroot /var/lib/haproxy
pidfile /var/run/haproxy.pid
maxconn 3000 #最大連接
user haproxy
group haproxy
daemon
# turn on stats unixsocket
stats socket/var/lib/haproxy/stats
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sectionswill
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http #工作模式爲http,有tcp等選擇,自行查看man文檔
log global #記錄日誌
option httplog #詳細記錄http日誌
option dontlognull #不記錄健康檢查的日誌信息
option http-server-close #啓用服務器端主動關閉
option forwardfor except 127.0.0.0/8 #傳遞客戶端IP
retries 3 #請求重試次數
timeout http-request 10s #http請求超時時間
timeout queue 1m #一個請求在隊列裏的超時時間
timeout connect 10s #連接服務器超時時間
timeout client 1m #客戶端超時時間
timeout server 1m #服務器超時時間
timeout http-keep-alive 10s #持久連接超時時間
timeout check 10s #心跳檢測超時時間
maxconn 3000 #最大連接
#---------------------------------------------------------------------
# main frontend which proxys to the backends
#---------------------------------------------------------------------
frontend main *:80
#acl url_static path_beg -i /static /images/javascript /stylesheets
#acl url_static path_end -i .jpg .gif .png.css .js
acl url_www path_end -i .php
use_backend www if url_www #如果上方啓用的acl中匹配任意,即使用www的後端服務器
default_backend static #默認請求連接發往static的後端服務器
#---------------------------------------------------------------------
# round robin balancing between the various backends
#---------------------------------------------------------------------
backend static #static後端服務器組定義
balance roundrobin #對static中定義的後端服務器,採用的調度算法爲roundrobin輪詢
server static1 192.168.43.112:80 check #後端服務器static1的IP,以及做健康狀態監測
server static1192.168.43.254:80 check
backend www #www後端服務器組定義
balance roundrobin
server app2 192.168.43.113:80 check
listen stats
bind *:8888 #狀態頁的訪問端口
stats enable #haproxy狀態頁定義
stats uri /stats #狀態頁的訪問路徑
stats realm please\ input\ passwd #提示消息,請輸入密碼,空格需要用\轉義纔可顯示
stats auth admin:liao #驗證用戶名:admin |密碼:liao
stats admin if TRUE #如果用戶名驗證成功,則顯示管理功能
2).修改keepalived配置文件
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
smtp_server192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_down { #定義腳本chk_down
script "[ -f /etc/keepalived/down ] && exit 1 || exit0" #若目錄存在down文件則返回1,不存在返回0
interval 1
weight -5 #若存在down文件則減去權重5
}
vrrp_script chk_haproxy { #定義腳本 chk_haproxy
script "killall -0 haproxy &>/dev/null" #檢測haproxy是否正常運行,若正常運行則會返回0,否則返回1,
interval 1 #1秒檢測一次
weight -5 #若檢測haproxy不是正常運行,那麼返回值爲1,減去權重5
}
vrrp_instance VI_1 { #第一個虛擬路由
state MASTER #運行爲主節點
interface eth0
virtual_router_id 51 #虛擬路由ID爲51
priority 100 #主節點權重爲100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.43.119/32 brd 192.168.43.119 dev eth0label eth0:0 #在eth0:0端口配置192.168.43.119爲虛擬地址
}
track_script { #調用上方定義的檢測腳本
chk_down
chk_haproxy
}
notify_master "/etc/keepalived/haproxy.sh master" #若狀態變爲主節點,則運行haproxy.sh腳本且附帶參數master(腳本在下方)
notify_backup "/etc/keepalived/haproxy.sh backup" #若狀態變爲備節點,則運行haproxy.sh腳本且附帶參數backup
notify_fault"/etc/keepalived/haproxy.sh fault"
}
vrrp_instance VI_2 { #虛擬路由器2
state BACKUP #運行爲備用節點
interface eth0
virtual_router_id 52 #虛擬路由器ID爲52
priority 99 #權重99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.43.120/32 brd 192.168.43.120 dev eth0label eth0:1 #在eth0:1上配置虛擬ip192.168.43.120
}
track_script { #一樣需要檢測上方定義的腳本
chk_down
chk_haproxy
}
notify_master"/etc/keepalived/haproxy.sh master"
notify_backup"/etc/keepalived/haproxy.sh backup"
notify_fault"/etc/keepalived/haproxy.sh fault"
}
3).創建上方調用的haproxy.sh腳本
# vim /etc/keepalived/haproxy.sh
#!/bin/bash
case "$1" in
master)
/etc/rc.d/init.d/haproxy start #若傳遞過來的參數爲master,則啓動haproxy
;;
backup)
/etc/rc.d/init.d/haproxy restart #若傳遞過來的參數爲backup,則重啓haproxy
;;
fault)
/etc/rc.d/init.d/haproxy stop #若傳遞過來的參數爲fault,則關閉haproxy
esac
4).將配置和腳本文件複製到b.com
# scp/etc/keepalived/keepalived.conf /etc/keepalived/haproxy.sh b.com:/etc/keepalived/
# scp/etc/haproxy/haproxy.cnf b.com:/etc/haproxy
5).切換到b.com操作,適當修改複製到b.com上的keepalived配置文件(只需修改紅色字體內容即可)
# vim/etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
smtp_server192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_down { #定義腳本chk_down
script "[ -f /etc/keepalived/down ] && exit 1 || exit0" #若目錄存在down文件則返回1,不存在返回0
interval 1
weight -5 #若存在down文件則減去權重5
}
vrrp_script chk_haproxy { #定義腳本 chk_haproxy
script "killall -0 haproxy &>/dev/null" #檢測haproxy是否正常運行,若正常運行則會返回0,否則返回1,
interval 1 #1秒檢測一次
weight -5 #若檢測haproxy不是正常運行,那麼返回值爲1,減去權重5
}
vrrp_instance VI_1 { #第一個虛擬路由
state BACKUP #運行爲主節點
interface eth0
virtual_router_id 51 #虛擬路由ID爲51
priority 99 #主節點權重爲100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.43.119/32 brd 192.168.43.119 dev eth0label eth0:0 #在eth0:0端口配置192.168.43.119爲虛擬地址
}
track_script { #調用上方定義的檢測腳本
chk_down
chk_haproxy
}
notify_master "/etc/keepalived/haproxy.sh master" #若狀態變爲主節點,則運行haproxy.sh腳本且附帶參數master(腳本在下方)
notify_backup "/etc/keepalived/haproxy.sh backup" #若狀態變爲備節點,則運行haproxy.sh腳本且附帶參數backup
notify_fault"/etc/keepalived/haproxy.sh fault"
}
vrrp_instance VI_2 { #虛擬路由器2
state MASTER #運行爲備用節點
interface eth0
virtual_router_id 52 #虛擬路由器ID爲52
priority 100 #權重99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.43.120/32 brd 192.168.43.120 dev eth0label eth0:1 #在eth0:1上配置虛擬ip192.168.43.120
}
track_script { #一樣需要檢測上方定義的腳本
chk_down
chk_haproxy
}
notify_master"/etc/keepalived/haproxy.sh master"
notify_backup"/etc/keepalived/haproxy.sh backup"
notify_fault"/etc/keepalived/haproxy.sh fault"
}
6).分別在兩臺節點上給予haproxy.sh腳本執行權限
# chmod +x/etc/keepalived/haproxy.sh ; ssh a.com 'chmod +x /etc/keepalived/haproxy.sh'
啓動服務,測試。兩節點都正常運行時:
1).啓動服務
# service haproxy start ;ssh a.com 'service haproxy start'
# service keepalived start; ssh a.com 'service keepalived start'
2).分別查看兩個節點日誌
a.com
b.com
3).查看兩個節點IP
a.com
b.com
4).測試調度
測試192.168.43.119,a.com節點
動態資源調度到了192.168.43.113
靜態資源輪詢調度
測試192.168.43.120,b.com節點
動態資源調度到了192.168.43.113
靜態資源輪詢調度
測試b.com節點down掉時候,b.com的資源是否遷移到a.com:
1).停掉b.com的keepalived
# servicekeepalived stop
2).觀察a.com的日誌
3).查看a.com上的IP
4).訪問測試119和120查看是否能正常調度
訪問192.168.43.119
靜態資源,輪詢
訪問動態資源,調度到113
訪問測試192.168.43.120
靜態資源,輪詢
訪問動態資源,調度到113
測試b.com上線,能否將資源搶奪回來:
1).啓動b.com的keepalived服務
# servicekeepalived start
2).查看兩節點的日誌(a.com變爲備節點,移除IP。b.com變爲主節點,添加IP)
a.com
b.com
3).訪問測試(省略)
至此,keepalived雙主模型的高可用haproxy完成。
-----------------------------------------------------------------------------------------------
下面爲keepalived+nginx的簡單過程
1).nginx作爲主流的web服務器,同樣也可以作爲反向代理服務器,用於負載均衡調度,代替haproxy,而且性能和haproxy相差無幾;
2).所有配置文件僅僅只需修改幾處,即可用作keepalived高可用nignx;
1.修改"/etc/keepalived/keepalived.conf"配置文件。(a.com節點,紅色字體即爲修改的部分)
# vim/etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
smtp_server192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_down { #定義腳本chk_down
script "[ -f /etc/keepalived/down ] && exit 1 || exit0" #若目錄存在down文件則返回1,不存在返回0
interval 1
weight -5 #若存在down文件則減去權重5
}
vrrp_script chk_nginx { #定義腳本 chk_nginx
script "killall -0 nginx&>/dev/null" #檢測nginx是否正常運行,若正常運行則會返回0,否則返回1,
interval 1 #1秒檢測一次
weight -5 #若檢測nginx不是正常運行,那麼返回值爲1,減去權重5
}
vrrp_instance VI_1 { #第一個虛擬路由
state MASTER #運行爲主節點
interface eth0
virtual_router_id 51 #虛擬路由ID爲51
priority 100 #主節點權重爲100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.43.119/32 brd 192.168.43.119 dev eth0label eth0:0 #在eth0:0端口配置192.168.43.119爲虛擬地址
}
track_script { #調用上方定義的檢測腳本
chk_down
chk_nginx
}
notify_master"/etc/keepalived/nginx.shmaster" #若狀態變爲主節點,則運行haproxy.sh腳本且附帶參數master(腳本在下方)
notify_backup "/etc/keepalived/nginx.shbackup" #若狀態變爲備節點,則運行haproxy.sh腳本且附帶參數backup
notify_fault "/etc/keepalived/nginx.sh fault"
}
vrrp_instance VI_2 { #虛擬路由器2
state BACKUP #運行爲備用節點
interface eth0
virtual_router_id 52 #虛擬路由器ID爲52
priority 99 #權重99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.43.120/32 brd 192.168.43.120 dev eth0label eth0:1 #在eth0:1上配置虛擬ip192.168.43.120
}
track_script { #一樣需要檢測上方定義的腳本
chk_down
chk_nginx
}
notify_master "/etc/keepalived/nginx.sh master"
notify_backup "/etc/keepalived/nginx.sh backup"
notify_fault "/etc/keepalived/nginx.sh fault"
}
2.創建/etc/keepalived/nginx.sh腳本(記得給予nginx.sh執行權限chmod +x nginx.sh)
腳本1
# vim /etc/keepalived/nginx.sh
#!/bin/bash
#nginx script
#我的nginx爲編譯安裝,/etc/rc.d/目錄下無服務腳本,使用此腳本。
case "$1" in
master)
/usr/local/nginx/sbin/nginx #啓動nginx
;;
backup)
/usr/local/nginx/sbin/nginx -s stop #先停止nginx
/usr/local/nginx/sbin/nginx #再啓動nginx,保證nginx無論主備節點都處在運行態,讓killall -0 nginx檢測成功
;;
fault)
/usr/local/nginx/sbin/nginx -s stop
;;
esac
腳本2
# vim /etc/keepalived/nginx.sh
#!/bin/bash
#nginx script
#rpm包安裝的nginx腳本
case "$1" in
master)
/etc/rc.d/init.d/nginx start
;;
backup)
/etc/rc.d/init.d/nginx restart
;;
fault)
/etc/rc.d/init.d/nginx stop
;;
esac
3.配置nginx
# vim/etc/nginx/nginx.conf
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent"$http_referer" '
# '"$http_user_agent""$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
upstream staticserver { #設置一個服務器組,不加權重爲rr調度算法,加權重爲wrr調度算法
server 192.168.43.112 weight 1;
server 192.168.43.254 weight 1;
#ip_hash; #加上ip_hash即爲源地址hash算法,在此處沒使用用,所以註釋了
}
server {
listen 80;
server_name www.a.com;
#charset koi8-r;
#access_log logs/host.access.log main;
#location~* \.php$ { #註釋掉的動態內容調度,若想做動靜分離的可以使用
# proxy_pass http://192.168.43.113:80;
#}
location / {
# root /var/www/html;
index index.php index.html index.htm;
proxy_pass http://staticserver; # 使用反向代理,將用戶請求調度到staticserver中定義的後端服務器
}
}
4.將所有上方配置的文件複製到b.com
# scp/etc/keepalived/keepalived.conf b.com:/etc/keepalived/
# scp/etc/nginx/nginx.conf b.com:/etc/nginx/
# scp/etc/keepalived/nginx.sh b.com:/etc/keepalived/
5.切換到b.com操作,修改一下複製過來的keepalived.conf配置文件(紅色字體爲修改部分)
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
}
smtp_server192.168.200.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_script chk_down { #定義腳本chk_down
script "[ -f /etc/keepalived/down ] && exit 1 || exit0" #若目錄存在down文件則返回1,不存在返回0
interval 1
weight -5 #若存在down文件則減去權重5
}
vrrp_script chk_nginx { #定義腳本 chk_nginx
script "killall -0 nginx&>/dev/null" #檢測nginx是否正常運行,若正常運行則會返回0,否則返回1,
interval 1 #1秒檢測一次
weight -5 #若檢測nginx不是正常運行,那麼返回值爲1,減去權重5
}
vrrp_instance VI_1 { #第一個虛擬路由
state BACKUP #運行爲備用節點
interface eth0
virtual_router_id 51 #虛擬路由ID爲51
priority 99 #主節點權重爲99
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.43.119/32 brd 192.168.43.119 dev eth0label eth0:0 #在eth0:0端口配置192.168.43.119爲虛擬地址
}
track_script { #調用上方定義的檢測腳本
chk_down
chk_nginx
}
notify_master "/etc/keepalived/nginx.shmaster" #若狀態變爲主節點,則運行haproxy.sh腳本且附帶參數master(腳本在下方)
notify_backup "/etc/keepalived/nginx.shbackup" #若狀態變爲備節點,則運行haproxy.sh腳本且附帶參數backup
notify_fault "/etc/keepalived/nginx.sh fault"
}
vrrp_instance VI_2 { #虛擬路由器2
state MASTER #運行爲主節點
interface eth0
virtual_router_id 52 #虛擬路由器ID爲52
priority 100 #權重100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.43.120/32 brd 192.168.43.120 dev eth0label eth0:1 #在eth0:1上配置虛擬ip192.168.43.120
}
track_script { #一樣需要檢測上方定義的腳本
chk_down
chk_nginx
}
notify_master "/etc/keepalived/nginx.sh master"
notify_backup "/etc/keepalived/nginx.sh backup"
notify_fault "/etc/keepalived/nginx.sh fault"
}
3).此時配置應該完成,在此就不做測試了