Haproxy+keepalivd+varnish+LAMP
目 錄
實驗要求
實驗前提
實驗環境設計
拓撲結構
ip網絡規劃
基礎環境配置
一、LAMP配置
1、HTTPD配置
2、Discuz安裝
二、Haproxy配置：
Haproxy日誌輸出配置
三、Keepalived配置
四、Varnish 配置
五、修改web服務器日誌配置
六、測試
1、客戶端訪問測試
2、客戶端訪問haprxoy監控頁面測試
3、varnish緩存命中查看
4、後端服務器日誌查看
5、重要 假設haproxy 不設置會話保持就會出現下面的情況
七、總結
實驗要求
1、 動靜分離部署Discuz，動靜要實現負載均衡，注意會話問題；
2、 Haproxy和後端主機間添加Vanish緩存；
3、 Hapeoxy設計要求
a、動靜分離；
b、stats page ，要求僅能通過本機主機使用管理接口；
c、粉筆考慮不同服務組的調度算法；
d、Haporxy高可用；
4、 給出拓撲，寫成博客；
實驗前提
實驗前需要關閉iptables及selinux；
實驗前做好時間同步
實驗環境設計
拓撲結構
根據以上要求本次實驗給出以下拓撲：
ip網絡規劃
根據拓撲需求以下爲各服務器網絡配置：
注：1、因筆者筆記本性能問題dynamic-web組與static-web組分別用一臺機器模擬；httpd虛擬主機實現；
2、本實驗環境未做mysql的高可用集羣，因此本實驗環境，將mysql安裝在172.16.76.30服務器僅作實驗環境需求；
客戶端ip：172.16.250.186；
Haproxy1：172.16.76.10；
Haproxy1：172.16.76.20；
KeepalivedVIP1:172.16.76.100；
KeepalivedVIP2:172.16.76.110;
Varnish:172.16.76.60;
dynamic-web1:172.16.76.30:80; dynamic-web1:172.16.76.30:8080;
static-web1:172.16.76.40:80；同上
static-web2:172.16.76.40:8080;
Mysql:172.16.76.30:3306;
注：以下主機名對應實驗環境響應服務器
node1:172.16.76.10;
node2:172.16.76.20;
node3:172.16.76.30;
node4:172.16:76.40;
node6:172.16.76.60;
基礎環境配置
node1：
[root@node1~]# yum install haproxy keepalived -y
node2:
[root@node2~]# yum install haproxy keepalived -y
node3:
[root@node3~]yum install mariadb-server httpd -y
node4:
[root@node4~]yum install httpd –y
node6:
[root@node6~]# yum install varnish -y
一、LAMP配置
1、HTTPD配置；
首先現在先配置後端web服務；
node3：
[root@node3 ~]# cd /etc/httpd/conf.d/ [root@node3 conf.d]# vim vhost.conf <VirtualHost 172.16.76.30:8080> DocumentRoot/apps/www/html/ ServerNamewww.linuxinfo.top ErrorLoglogs/bbs-8080.error_log Customloglogs/bbs-8080.access_log combined <Directory"/apps/www/html/"> Options None AllowOverrideNone Require allgranted </Directory> </VirtualHost> <VirtualHost 172.16.76.30:80> DocumentRoot/var/www/html/ ServerNamewww.linuxinfo.top ErrorLoglogs/bbs.error_log Customloglogs/bbs.access_log combined #單獨記錄日誌 <Directory"/var/www/html/"> Options None AllowOverrideNone Require all granted </Directory> </VirtualHost> [root@node3 conf.d]# vim ../conf/httpd.conf Listen 80 Listen 8080 #設置Httpd監聽端口； #DocumentRoot "/var/www/html" #將該行註釋； [root@node3 conf.d]# systemctl restarthttpd.service [root@node3 ~]# mysql MariaDB [(none)]>grantall on test.* to test@'localhost' identified by "test"; MariaDB [(none)]> grant all on test.*to test@'127.0.0.1' identified by "test"; MariaDB [(none)]> grant all on test.* totest@'172.16.76.40' identified by "test"; #此處允許172.16.76.40訪問。
node4:
[root@node4 ~]# cd /etc/httpd/conf.d/ [root@node4 conf.d]# vim vhost.conf <VirtualHost 172.16.76.40:8080> DocumentRoot/apps/www/html/ ServerNamewww.linuxinfo.top ErrorLoglogs/bbs-8080.error_log Customloglogs/bbs-8080.access_log combined <Directory"/apps/www/html/"> Options None AllowOverrideNone Require all granted </Directory> </VirtualHost> <VirtualHost 172.16.76.40:80> DocumentRoot/var/www/html/ ServerNamewww.linuxinfo.top ErrorLoglogs/bbs.error_log Customloglogs/bbs.access_log combined <Directory"/var/www/html/"> Options None AllowOverrideNone Require all granted </Directory> </VirtualHost> [root@node4 conf.d]# vim ../conf/httpd.conf Listen 80 Listen 8080 #設置Httpd監聽端口； #DocumentRoot "/var/www/html" #將該行註釋； [root@node4 conf.d]# systemctl restart httpd.service
2、Discuz安裝
[root@node3 ~]#cd /var/www/html/ [root@node3 ~]#unzip Discuz_X3.3_SC_UTF8_0101.zip -d html [root@node3 ~]# chown -Rapache.apache html/
Discuz在172.16.16.30:80安裝成功；因172.16.76.30還配置虛擬主機8080；
所以此處將已安裝好的Discuz的html目錄複製到8080虛擬主機即可；
[root@node3 ~]#cp –R /var/www/html /apps/www/html
爲保證node4上static-web組網站配置相同將node3完整html目錄打包傳遞到node4；
node3
root@node3 ~]#cd /var/www/ [root@node3 ~]#tar –cvf html.tar html/ [root@node3 ~]#scp html.tar 172.16.76.40:/var/www/ [root@node3 ~]#scp html.tar 172.16.76.40:/apps/www/
node4：
[root@node4 ~]#cd /var/www/
[root@node4 www]# tar –xvf html.tar
[root@node4 www]# cd /apps/www
[root@node4 www]# tar –xvf html.tar
注:因整個完整目錄整體從172.16.76.30拷貝而來此處，所以需要修改mysql的連接地址；
[root@node4 ~]#cd /var/www/
[root@node4 upload]# vim config/config_global.php
$_config['db']['1']['dbhost'] = '172.16.76.30'; #此處設置mysql地址；
[root@node4 upload]# vim config/config_ucenter.php
define('UC_DBHOST', '172.16.76.30');
[root@node4 upload]# vim uc_server/data/config.inc.php
define('UC_DBHOST', '172.16.76.30');
即此處共計修改三個文件：
config/config_global.php
config/config_ucenter.php
uc_server/data/config.inc.php
同上: 172.16.76.40:8080 虛擬主機目錄/apps/www/ 也需要修改以上三個文件。
至此LAMP全部配置完成； 寫到這筆者想起來在172.16.76.30服務器上安裝Discuz時其實可以配置數據庫地址爲172.16.76.30的這樣後續複製網站目錄時就不用修改那三個文件了
；
二、Haproxy配置：
node1：
[root@node1~]# cd /etc/haproxy/ root@node1 haproxy]# vim haproxy.cfg global log 127.0.0.1 local2 info #日誌配置 chroot /var/lib/haproxy pidfile /var/run/haproxy.pid maxconn 4000 user haproxy group haproxy daemon # turn on stats unixsocket stats socket/var/lib/haproxy/stats defaults mode http log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 10s timeout client 1m timeout server 1m timeout http-keep-alive10s timeout check 10s maxconn 3000 frontend www bind *:80 mode http option forwardfor header Client-IP log global 配置:以.php結尾所有url調度至webser服務器組即:dyanamc-web組; 以.css .js等結尾與usrl中包含/static/imagesd等字符的請求調至webser-static 即：static-web服務器組中； acl web url_end -i .php acl url_static url_beg -i /static /images /javascript acl url_static url_reg -i .js.*$ .css.*$ .jpg.*$ .png.*$.gif.*$ use_backend webser if web use_backend webser-static ifurl_static default_backend webser backend webser mode http option redispatch option abortonclose option forwardfor header Client-IP #將真實ip轉發至後端web；http日誌設置 balance source #source調度算法；會話保持（注：看最後測試） cookie SERVERID option httpchk GET/upload/forum.php server web1 172.16.76.30:80 cookie server1 weight 6 check inter 2000 \ rise 2 fall 3 server web2 172.16.76.30:8080 cookie server1 weight 6 check inter 2000 \ rise 2 fall 3 backend webser-static server web1 172.16.76.60:80 check inter2000 rise 2 fall 3 listen admin_stats bind *:9188 mode http stats refresh 30s stats uri/haproxy-status #配置監控頁面url stats realm welcomelogin\ Haproxy stats authadmin:adminpass #監控界面驗證用戶密碼 stats hide-version stats admin if TRUE acl client src 172.16.250.186 block unless client #此處配置除172.16.250.186ip之外其餘ip禁止訪問監控頁面； [root@node1haproxy]# systemctl restart haproxy 重啓haproxy； 因Haproxy配置高可用間此配置文件拷貝至node2即可； [root@node1 haproxy]#scp haproxy.cfg172.16.76.20:/etc/haproxy
node2:
[root@node2~]#systemctl restart haproxy
Haproxy日誌輸出配置
1、node1
[root@node1 ~]# vim /etc/rsyslog.d/haprxoy.conf $Modload imudp $UDPServerRun 514 local2.* /var/log/haproxy/haproxy_info.log [root@node1 ~]# vim /etc/sysconfig/rsyslog SYSLOGD_OPTIONS="-c 2 -r -m 0" #配置接收遠程日誌
2、node2同上
三、Keepalived配置
node1：
[root@node1 ~]# cd /etc/keepalived/
[root@node1 keepalived]# cat keepalived.conf
! Configuration File for keepalived
global_defs {
notification_email {
root@localhost
}
notification_email_fromkeepadmin@localhost
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id node1
vrrp_mcast_group4224.0.76.100
}
vrrp_script check_haproxy {
script "killall -0haproxy"
interval 2
weight 21
}
vrrp_instance HAporxy_VIP1 {
state MASTER # haproxy2 即node2上爲BACKUP
interface eth0
virtual_router_id 100
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass test1111
}
virtual_ipaddress {
172.16.76.100/16 deveth0 label eth0:0
}
notify_master"/etc/keepalived/notify.sh master VIP1 vrid100"
notify_backup"/etc/keepalived/notify.sh backup VIP1 vrid100"
notify_fault"/etc/keepalived/notify.sh fault VIP1 vrid100"
track_script {
check_haproxy
}
}
vrrp_instance HAprxoy_VIP2 {
state BACKUP # haproxy2 即node2上爲MASTER
interface eth0
virtual_router_id 110
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass 1111test
}
virtual_ipaddress {
172.16.76.110/16 deveth0 label eth0:1
}
notify_master"/etc/keepalived/notify.sh master VIP2 vrid110"
notify_backup"/etc/keepalived/notify.sh backup VIP2 vrid110"
notify_fault"/etc/keepalived/notify.sh fault VIP2 vrid110"
track_script {
check_haproxy
}
}
notify.sh腳本
notify.sh 腳本
#!/bin/bash
#
info="$2$3"
contact='root@localhost'
notify(){
mailsubject="$(hostname) $info tobe $1, vip floating"
mailbody="$(date +'%F %T'): vrrptransition, $(hostname) $info changed to be $1"
echo "$mailbody" | mail -s"$mailsubject" $contact
}
case$1 in
master)
notify master $2 $3
;;
backup)
notify backup $2 $3
;;
fault)
notify fault $2 $3
;;
*)
echo "Usage: $(basename$0) {master|backup|fault}"
exit 1
;;
esac將node1Keepalive的配置文件拷貝至node2一份（注:看配置文件註釋修改）
[root@node1~]# cd /etc/keepalived/
[root@node1 keepalived]# scp keepalived.conf172.16.76.20:/etc/keepalived/
node2
一次啓動node1 、 node2 Keepalived
[root@node1 ~]# systemctl restart Keepalived
[root@node2 ~]# systemctl restart Keepalived
四、Varnish 配置：
node6：
[root@node6~]# cd /etc/varnish/
[root@node6 varnish]# cat default.vcl
vcl 4.0;
import directors;
probe static {
.url ="/upload/forum.php";
.interval = 5s;
.timeout = 1s;
}
backend web1 {
.host ="172.16.76.50";
.port = "8080";
.probe = static;
}
backend web2 {
.host ="172.16.76.40";
.port = "8080";
.probe = static;
}
sub vcl_init {
new webser =directors.round_robin();
webser.add_backend(web1);
webser.add_backend(web2);
}
acl purgers {
"172.16.76.30"/32;
"172.16.76.40"/32;
}
sub vcl_purge {
return(synth(200,"Purged"));
}
sub vcl_recv {
if (req.restarts == 0) {
if(req.http.x-forwarded-for) {
setreq.http.X-Forwarded-For = req.http.X-Forwarded-For +"," +client.ip;
} else {
setreq.http.X-Forwarded-For = client.ip;
}
}
set req.backend_hint =webser.backend();
if (req.url ~ "\.(html|jpg|png|bmp|jpeg|gif|js|ico|swf|css)$") {
unset req.http.cookie;
}
if (req.method == "PURGE") {
if (!client.ip ~ purgers) {
return(synth(405,"Purgingnot allowed for " + client.ip));
}
return(purge);
}
}
sub vcl_backend_response {
}
sub vcl_deliver {
if (obj.hits>0) {
set resp.http.X-Cache ="Hit Via" + " " + server.ip;
}else {
set resp.http.X-Cache ="Miss Via" + " " +server.ip;
}
}
五、修改web服務器日誌配置
node3:
[root@node3~]# vim /etc/httpd/conf/httpd.conf
LogFormat"%{Client-ip}i %l %u %t \"%r\" %>s %b\"%{Referer}i\" \"%{User-Agent}i\"" combined
node4:同上
重啓啓動各個服務即可
六、測試
1、客戶端訪問測試
2、客戶端訪問haprxoy監控頁面測試
非客戶端172.16.76.186訪問測試
3、varnish緩存命中查看
4、後端服務器日誌查看
5、重要 假設haproxy 不設置會話保持就會出現下面的情況
驗證碼輸入正確，但是無法登錄。這是因爲session會話的原因；
七、總結
本實驗完成實現以下功能：
1、Haproxy負載均衡
2、Haproxy 監控頁面訪問控制
3、varnish緩存
4、動靜分離
5、會話保持
本實驗至此全部完成。實驗中可能會有下瑕疵，各位注意觀察出錯可別怪我沒有提醒哈；
若實驗中真有問題歡迎指正；