最近要上線一個新項目，需要軟負載，本來決定用nginx做反向代理，但裝完nginx以後，發現nginx無法顯示css、js，目前還在研究中，但由於項目較急，所以依然採用了經典架構LVS+Keepalived來實現。
環境：
LVS-01 192.168.199.117 MASTER
LVS-02 192.168.199.118 BACKUP
WEB-01 192.168.199.113
WEB-02 192.168.199.114
VIP 192.168.199.119
編譯安裝LVS
ln -s /usr/src/kernels/2.6.18-308.el5-x86_64/ /usr/src/linux
wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz
tar zxf ipvsadm-1.26.tar.gz
cd ipvsadm-1.26
make
發現報錯，報錯結果如下：
make[1]: *** [libipvs.o] Error1
make[1]: Leaving directory`/root/tools/ipvsadm-1.26/libipvs'
make: *** [libs] Error 2
經查詢，有好多種說法，均沒有解決我的問題：
1、 有的說是需要做內核軟鏈接 ln-s /usr/src/kernels/2.6.18-308.el5-x86_64/ /usr/src/linux
2、 有的說是要yuminstall gcc-*
3、還有的說是yum installlibnl* popt*
以上方法可能適用於其實環境，但是在我的機器上面均無效。折騰了好久，請教高手，又是查看http://www.linuxvirtualserver.org/Documents.html官方說明文檔，最終認爲版本與內核不匹配，總結爲以下2點：
1、 ipvsadm-1.26版本適合kernel 2.6.28-rc3及以後
2、 選擇好相應的ipvsadm後，還要選擇相對應的keepalived，否則會出現啓動keepalived後，沒有realserver列表。大致顯示就是這樣：
IP Virtual Server version 1.2.1(size=4096)
Prot LocalAddress:PortScheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
好了，問題找到了，重新操作，下載低版本的ipvsadm，我的內核版本爲2.6.18-308.el5-x86_64，用ipvsadm-1.24測試正常，keepalived的版本爲keepalived-1.1.15，具體操作如下：
安裝LVS
ln -s /usr/src/kernels/2.6.18-308.el5-x86_64/ /usr/src/linux
wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
tar zxvf ipvsadm-1.24.tar.gz
cd ipvsadm-1.24
make
make install
cd ..
ipvsadm –version
安裝keepalived
wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz
tar zxfkeepalived-1.1.15.tar.gz
cd keepalived-1.1.15
./configure
make
make install
cp /usr/local/etc/rc.d/init.d/keepalived /etc/init.d/
cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
mkdir /etc/keepalived/
cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/sbin/keepalived /usr/sbin/
/etc/init.d/keepalived start
至此，安裝完畢，下面配置文件。
LVS-01上面keepalived的配置文件，主與備基本一致，不同的爲紅色部分
vim /etc/keepalived/keepalived.conf
! Configuration File forkeepalived
global_defs {
router_id LVS-01 #routeid
}
vrrp_instance LVS {
state MASTER #備上爲BACKUP
interface eth0
virtual_router_id 251
priority 100 #備節點上比主的數值要小
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.199.119
}
}
virtual_server 192.168.199.11980 {
delay_loop 6
lb_algo wrr
lb_kind DR
persistence_timeout 60 #會話保持
protocol TCP
real_server 192.168.199.113 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
real_server 192.168.168.199.114 80 {
weight 1
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 80
}
}
}
啓動keealived
/etc/init.d/keepalived restart
檢查是否有VIP
[root@localhost ipvsadm-1.26]#ip add
1: lo:<LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:70:47:f4 brdff:ff:ff:ff:ff:ff
inet 192.168.199.117/24 brd 192.168.199.255scope global eth0
inet 192.168.199.119/32 scope global eth0
inet6 fe80::20c:29ff:fe70:47f4/64 scopelink
valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
檢查是否有realserver列表
[root@localhost ipvsadm-1.26]#ipvsadm -Ln
IP Virtual Server version 1.2.1(size=4096)
Prot LocalAddress:PortScheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.199.119:80 wrr
-> 192.168.199.113:80 Route 1 0 0
-> 192.168.199.114:80 Route 1 0 0
最後別忘了，有realserver上面配置VIP和ARP抑制，這個用腳本就好啦，具體如下：
[root@localhost tools]# cat realserver.sh
#!/bin/sh
[ -f"/etc/init.d/functions" ] && . /etc/init.d/functions
VIP=(
192.168.199.119
)
start(){
#bind VIP
for ((i=0;i<${#VIP[*]};i++))
do
/sbin/ifconfig lo:$i ${VIP[$i]} netmask255.255.255.255 up >/dev/null2>&1
done
#ARP restrain
echo "1">/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2">/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1">/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2">/proc/sys/net/ipv4/conf/all/arp_announce
}
stop(){
for ((i=0;i<${#VIP[*]};i++))
do
/sbin/ifconfig lo:$i ${VIP[$i]} netmask255.255.255.255 down
done
echo "0">/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0">/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0">/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0">/proc/sys/net/ipv4/conf/all/arp_announce
}
case "$1" in
start)
start
if [ $? -eq 0 ];then
action "ipvs start sucess" /bin/true
else
action "ipvs start failed "/bin/false
fi
;;
stop)
stop
if [ $? -eq 0 ];then
action "ipvs stop sucess" /bin/true
else
action "ipvs stop failed "/bin/false
fi
;;
*)
echo "USAGE:$0{start|stop}"
esac
執行完成後，查看VIP
[root@localhost tools]# ip add
1: lo:<LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet 192.168.199.119/32 brd 192.168.199.119scope global lo:0
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast qlen 1000
link/ether 00:0c:29:41:7c:7e brdff:ff:ff:ff:ff:ff
inet 192.168.199.114/24 brd 192.168.199.255scope global eth0
inet6 fe80::20c:29ff:fe41:7c7e/64 scopelink
valid_lft forever preferred_lft forever
3: sit0: <NOARP> mtu 1480qdisc noop
link/sit 0.0.0.0 brd 0.0.0.0
剩下的就是故障模擬與測試了，在此就不多寫了。