lvs-dr實現:
1個Director + 2個Real Server:
在lvs-dr類型的集羣中,各個主機(包括Director和各RS)都需要配置VIP;爲了解決IP地址衝突的問題,通常有以下幾種方法:
1.在前端路由器上靜態綁定VIP和MAC地址的對應關係;
2.在各個RS中使用arptables對ARP報文進行過濾;
3.在各個RS中修改對應的內核參數,以此來限制ARP報文的通告和應答級別;
arp_ignore
0:默認值;
1:
2:
arp_announce
0:默認值;
1:
2:
常用的內核參數設定值的選擇:
arp_ignore = 1
arp_announce =2
lvs-dr實現的簡單示例:
三臺虛擬機
Director(CentOS 7.2A):
DIP:172.16.72.1
//在eno16777736網卡接口上
VIP:172.16.72.254
//在eno16777736網卡接口的label(標籤)上
Real Server1(CentOS 7.2B):
DIP:172.16.72.2
//在eno16777736網卡接口上
VIP:172.16.72.254
//在lo(環回接口)的label(標籤)上
Real Server2(CentOS 7.2C):
DIP:172.16.72.3
//在eno16777736網卡接口上
VIP:172.16.72.254
//在lo(環回接口)的label(標籤)上
1.修改對應的主機名
Director(CentOS 7.2A)
~]# hostnamectl set-hostname drct1
Real Server1(CentOS 7.2B)
~]# hostnamectl set-hostname rs1
Real Server2(CentOS 7.2C)
~]# hostnamectl set-hostname rs2
2.在Director(CentOS 7.2A)查看DIP,在DIP對應的網絡接口上設置VIP
~]# ifconfig
~]# ifconfig eno16777736:0 172.16.72.254 netmask 255.255.255.255 broadcast 172.16.72.254 up
3.在RS1(CentOS 7.2B)、RS2(CentOS 7.2C)上利用腳本進行的配置;
#!/bin/bash
#
VIP=172.16.72.254
MASK=255.255.255.255
case $1 in
setup)
#調整ARP相關的內核參數:
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
#在lo的標籤接口上配置VIP
ifconfig lo:0 $VIP netmask $MASK broadcast $VIP up
#爲了能夠使響應報文從lo:0標籤接口向外封裝發送數據,需要指定一條特殊的靜態路由:
route add -host $VIP dev lo:0
;;
delete)
ifconfig lo:0 down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
;;
*)
echo "Usage: $(basename $0) { setup | delete }"
;;
esac
4.向Director(CentOS 7.2A)的集羣服務添加集羣RS:
~]# ipvsadm -A -t 172.16.72.254:80 -s rr
~]# ipvsadm -E -t 172.16.72.254:80 -s wrr
~]# ipvsadm -a -t 172.16.72.254:80 -r 172.16.72.2 -g -w 1
~]# ipvsadm -a -t 172.16.72.254:80 -r 172.16.72.3 -g -w 2
~]# ipvsadm -l
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.72.254:http rr
-> 172.16.72.2:http Route 1 0 0
-> 172.16.72.3:http Route 2 0 0
5.在客戶端(CentOS 7.2D)測試
~]# for i in {1..10}; do curl http://172.16.72.254 ;done
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
This is CentOS 7.2B for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
This is CentOS 7.2B for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
This is CentOS 7.2B for /var/www/html/
this is CentOS 7.2C for /var/www/html/
使用FWM(Firewall Mark)的方式定義集羣服務:
優勢:可以實現多個服務的集羣服務同時定義;
在netfilter上,爲某些匹配規則所匹配到的數據報文添加對應的標記;因此需要在mangle表上進行設置;爲了配合工作於INPUT鏈上的ipvs的工作,只能在netfilter的PREROUTING鏈上對數據報文進行標記;
1.打標記的方法:
~]# ipvsadm -C
~]# iptables -t mangle -A PREROUTING -d 172.16.72.254 -p tcp --dport 80 -j MARK --set-mark 15
或者
~]# iptables -t mangle -A PREROUTING -d 172.16.72.254 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
2.lvs集羣定義方法:lvs-dr類型實現:
~]# ipvsadm -A -f 15 -s wrr
~]# ipvsadm -a -f 15 -r 172.16.72.3 -g -w 3
~]# ipvsadm -a -f 15 -r 172.16.72.2 -g -w 1
~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 15 wrr
-> 172.16.72.2:0 Route 1 0 0
-> 172.16.72.3:0 Route 3 0 0
3.客戶端(CentOS 7.2D)測試
~]# for i in {1..10}; do curl http://172.16.72.254 ;done
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
This is CentOS 7.2B for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
This is CentOS 7.2B for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
lvs persistence:lvs的持久連接;
當客戶端和RS建立連接時,會創建一個持久連接的模版;基於此模版實現無論使用任何調度算法,都會在一段時間內將來自於同一個源IP地址的請求始終調度至後端同一臺RS上,只有第一次調度是根據算法來進行選擇;
可以將持久鏈接理解爲:帶有時間限制的SH算法;
三種可以選擇的持久連接的方案:
1.每端口持久連接:僅在一段時間內,將來自於同一源IP地址的訪問某一特定服務的請求調度轉發至後端的同一臺RS上;
調度標準:VIP:PORT
2.每客戶端持久連接:僅在一段時間內,將來自於同一源IP地址的訪問請求,統一調度至後端同一臺RS上,更像是SH算法的應用;
調度標準:VIP:0
3.每防火牆標記持久連接:僅在一段時間內,將對於綁定在同一個FWM的所有請求,調度至後端同一臺RS;
調度標準;FWM
持久連接+防火牆標記:端口姻親關係(Port Affinity)
最常見的端口姻親關係就是:80 + 443
1.做防火牆標記,在基於防護牆標記建立lvs集羣服務並開啓持久連接:
~]# iptables -t mangle -A PREROUTING -d 172.16.72.254 -p tcp --dport 80 -j MARK --set-mark 15
或者
~]# iptables -t mangle -A PREROUTING -d 172.16.72.254 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
2.lvs集羣定義方法:lvs-dr類型實現,添加持久連接時長:
~]# ipvsadm -A -f 15 -s wrr
~]# ipvsadm -a -f 15 -r 172.16.72.3 -g -w 3
~]# ipvsadm -a -f 15 -r 172.16.72.2 -g -w 1
~]# ipvsadm -E -f 15 -s wrr -p 30
//30s的持久連接時長,不寫-p後面的值默認360秒
~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 15 wrr persistent 30
-> 172.16.72.2:0 Route 1 0 0
-> 172.16.72.3:0 Route 3 0 0
3.客戶端(CentOS 7.2D)測試
首次測試:
~]# for i in {1..10}; do curl http://172.16.72.254 ;done
This is CentOS 7.2B for /var/www/html/
This is CentOS 7.2B for /var/www/html/
This is CentOS 7.2B for /var/www/html/
This is CentOS 7.2B for /var/www/html/
This is CentOS 7.2B for /var/www/html/
This is CentOS 7.2B for /var/www/html/
This is CentOS 7.2B for /var/www/html/
This is CentOS 7.2B for /var/www/html/
This is CentOS 7.2B for /var/www/html/
This is CentOS 7.2B for /var/www/html/
等待一段時間過後測試:
~]# for i in {1..10}; do curl http://172.16.72.254 ;done
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/
this is CentOS 7.2C for /var/www/html/