NginxUpStream模塊三種綁定模默認wrr ip_hash 以及hashkey的實現

實驗要求，至少三臺虛擬機，在同一網段172.18/16，172.18.10.10/11，其中172.18.200.100做爲DR

並且配置兩塊網卡，另一塊網卡的地址不能再171.18/16網段內

[root@localhost ~]# ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 

    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host 

       valid_lft forever preferred_lft forever

2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:23:f3:8d brd ff:ff:ff:ff:ff:ff

    inet 172.18.200.100/16 brd 172.18.255.255 scope global eth1

    inet6 fe80::20c:29ff:fe23:f38d/64 scope link 

       valid_lft forever preferred_lft forever

3: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

    link/ether 00:0c:29:23:f3:97 brd ff:ff:ff:ff:ff:ff

    inet 192.168.10.100/24 brd 192.168.10.255 scope global eth2

    inet6 fe80::20c:29ff:fe23:f397/64 scope link 

       valid_lft forever preferred_lft forever

1、在10/11上使用yum安裝php和httpd

[root@localhost ~]# yum install php httpd

2、配置時間同步，先安裝chrony安裝包，再進行配置

[root@localhost ~]# yum install -y chrony

[root@localhost ~]# vim /etc/chrony.conf

# Allow NTP client access from local network.

#allow 192.168/16

allow 172.18/16

# Serve time even if not synchronized to any NTP server.

local stratum 10

3、啓動chrony服務

[root@localhost ~]# service chronyd start

Starting chronyd:                                          [  OK  ]

4、安裝同步時間的守護進程，爲的是每過一段時間同步時間

分別在兩臺Vs上安裝chrony

[root@localhost ~]# yum install -y chrony

[root@localhost ~]# yum install -y chrony

編輯配置文件,兩臺VS機子一樣的配置

[root@localhost ~]# vim /etc/chrony.conf

#server 0.rhel.pool.ntp.org iburst

#server 1.rhel.pool.ntp.org iburst

#server 2.rhel.pool.ntp.org iburst

#server 3.rhel.pool.ntp.org iburst

server 172.18.200.100 iburst

啓動chrony服務，兩臺VS機子一樣的配置

[root@localhost ~]# service chronyd start

Starting chronyd:                                          [  OK  ]

5、與服務器同步時間

[root@localhost ~]#  ntpdate 172.18.200.100

10 May 02:13:57 ntpdate[3324]: step time server 172.18.200.100 offset -2587.052960 sec

[root@localhost ~]#  ntpdate 172.18.200.100

10 May 02:13:57 ntpdate[3324]: step time server 172.18.200.100 offset -2587.052960 sec

[root@localhost ~]# date

Wed May 10 02:31:05 CST 2017

[root@localhost ~]# date

Wed May 10 02:31:08 CST 2017

6、在兩臺VS主機上提供相關主頁

這裏用簡單的for循環快速生成10個頁面測試文件

[root@localhost ~]# for i in {1..20}; do echo "Test Page $i on UpStream Server 1 (172.18.10.10)" > /var/www/html/test$i.html;done

[root@localhost html]# ls /var/www/html/

test10.html  test12.html  test14.html  test16.html  test18.html  test1.html   test2.html  test4.html  test6.html  test8.html

test11.html  test13.html  test15.html  test17.html  test19.html  test20.html  test3.html  test5.html  test7.html  test9.html

另一臺機子做同樣的操作

7、啓動httpd服務（10/11）

root@localhost html]# service httpd start

8、在DR服務端測試，使用curl測試頁面是否能訪問

[root@localhost ~]# curl http://172.18.10.10/test1.html

Test Page 1 on UpStream Server 1 (172.18.10.10)

[root@localhost ~]# curl http://172.18.10.11/test1.html

Test Page 1 on UpStream Server 1 (172.18.10.11)

9、在DR端下載並安裝nginx，由於使用centos6.8,無法安裝nginx1.10以上的版本

lftp 172.18.0.1:/pub/Sources/6.x86_64/nginx> ls

-rw-r--r--    1 500      500        714233 Jul 25  2013 nginx-1.0.15-5.el6.src.rpm

-rwxr--r--    1 500      500        319456 Apr 24  2014 nginx-1.4.7-1.el6.ngx.x86_64.rpm

-rw-r--r--    1 0        0          344416 Sep 16  2014 nginx-1.6.2-1.el6.ngx.x86_64.rpm

lftp 172.18.0.1:/pub/Sources/6.x86_64/nginx> mget nginx-1.6.2-1.el6.ngx.x86_64.rpm 

344416 bytes transferred

lftp 172.18.0.1:/pub/Sources/6.x86_64/nginx> bye

安裝nginx

[root@localhost ~]# yum install nginx-1.6.2-1.el6.ngx.x86_64.rpm

10、配置nginx，僅僅作爲反向代理服務器

[root@localhost ~]# cd /etc/nginx/conf.d/

[root@localhost conf.d]# ls

default.conf  example_ssl.conf

[root@localhost conf.d]# cp default.conf default.conf.bak

[root@localhost conf.d]# ls

default.conf  default.conf.bak  example_ssl.conf

因爲要先定義組，所以首先編輯nginx.conf文件

[root@localhost nginx]# ls

conf.d  fastcgi_params  koi-utf  koi-win  mime.types  nginx.conf  scgi_params  uwsgi_params  win-utf

[root@localhost nginx]# vim nginx.conf 

在http上下文中定義upstream 模塊，操作如下

http {

    include       /etc/nginx/mime.types;

    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '

                      '$status $body_bytes_sent "$http_referer" '

                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;

    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

        upstream websrvs {

                server 172.18.10.10:80;

                server 172.18.10.11:80;

        }

    include /etc/nginx/conf.d/*.conf;

}編輯conf.d文件中的default.conf文件

[root@localhost conf.d]# vim default.conf

在location上下文中編輯proxy_pass反向代理文件

server {

    listen       80;

    server_name  localhost;

    #charset koi8-r;

    #access_log  /var/log/nginx/log/host.access.log  main;

    location / {

        proxy_pass http://websrvs;

        root   /usr/share/nginx/html;

        index  index.html index.htm;

nginx -t 檢查配置文件語法

10、啓動nginx服務

[root@localhost conf.d]# nginx

查看監聽端口80是否開啓

[root@localhost conf.d]# ss -tnl

State      Recv-Q Send-Q                                 Local Address:Port                                   Peer Address:Port 

LISTEN     0      128                                                *:80                                                *:*     

LISTEN     0      128                                               :::22                                               :::*     

LISTEN     0      128                                                *:22                                                *:*     

LISTEN     0      100                                              ::1:25                                               :::*     

LISTEN     0      100                                        127.0.0.1:25 

11、在客戶端上使用curl命令一次性訪問10次頁面，看看效果

[root@localhost ~]# for ((i=1;i<=10;i++));do curl http://172.18.200.100:80/test1.html; done

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

發現實現負載均衡

12、若期望負載權限有所不同，則進行如下配置

[root@localhost conf.d]# cd ..

[root@localhost nginx]# vim nginx.conf 

    #gzip  on;

        upstream websrvs {

                server 172.18.10.10:80 weigth=2;

                server 172.18.10.11:80 weight=3; #### 這裏權重：綜合2+3，意爲第一臺服務器虛擬爲2臺，第二臺服務器虛擬爲3臺

在upstream 模塊裏增加權重設置

保存退出，重啓服務

[root@localhost nginx]# nginx -t

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conf test is successful

[root@localhost nginx]# nginx -s reload

13、客戶端再次訪問10次效果如下

[root@localhost ~]# for ((i=1;i<=10;i++));do curl http://172.18.200.100:80/test1.html; done

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

14、關閉其中一臺VS，再次測試

[root@localhost ~]# service httpd stop

Stopping httpd:                                            [  OK  ]

[root@localhost ~]# for ((i=1;i<=10;i++));do curl http://172.18.200.100:80/test1.html; done

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

再次啓動VS,在測試

[root@localhost ~]# service httpd start

[root@localhost ~]# for ((i=1;i<=10;i++));do curl http://172.18.200.100:80/test1.html; done

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

15、定義最大失敗次數max_fails ,和超時時間間隔 fail_timeout

[root@localhost nginx]# vim nginx.conf 

        upstream websrvs {

                server 172.18.10.10:80 weight=2 max_fails=2 fail_timeout=2;

                server 172.18.10.11:80 weight=3 backup;

保存退出並重啓服務

在客戶端測試

[root@localhost ~]# for ((i=1;i<=10;i++));do curl http://172.18.200.100:80/test1.html; done

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

再次修改配置

[root@localhost ~]# for ((i=1;i<=10;i++));do curl http://172.18.200.100:80/test1.html; done

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

因此發現nginx默認調度規則爲加權輪詢wrr

16、實現ip_hash（ip哈希綁定）綁定,第一次訪問哪個主機，隨後都會訪問該主機

vim nginx.conf

 upstream websrvs {

                ip_hash;

                server 172.18.10.10:80 weight=2 max_fails=2 fail_timeout=2;

                server 172.18.10.11:80 weight=3;

[root@localhost ~]# for ((i=1;i<=10;i++));do curl http://172.18.200.100:80/test1.html; done

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 1 (172.18.10.10)

17、實現least_conn（加權最少連接），相當於wlc，最少連接，只要weight不同就會考慮權重

   upstream websrvs {

                least_conn;

                server 172.18.10.10:80 weight=2 max_fails=2 fail_timeout=2;

                server 172.18.10.11:80 weight=3;

        }

[root@localhost ~]# for ((i=1;i<=10;i++));do curl http://172.18.200.100:80/test1.html; done

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

Test Page 1 on UpStream Server 1 (172.18.10.10)

Test Page 1 on UpStream Server 2 (172.18.10.11)

18.哈希 hash key ，key可以是任何表示，url，地址等等，hash後跟什麼就綁定什麼

     upstream websrvs {

                hash $request;

                server 172.18.10.10:80 weight=2 max_fails=2 fail_timeout=2;

                server 172.18.10.11:80 weight=3;

        }

第一次訪問那個資源在哪臺服務器，之後就綁定再該服務器上

upstream websrvs {

                hash $request consistent;   #####一致性哈希算法

                server 172.18.10.10:80 weight=2 max_fails=2 fail_timeout=2;

                server 172.18.10.11:80 weight=3;

        }

19、keepalive connections

併發太高，給端口帶來壓力，爲此避免這種情況

可以在後面這一側，使用長連接

nginx的特點，不是每一個請求靠每一個進程來響應，

而是一個進程響應n個請求

因此大量請求都是由worker響應的

worker 4個

現在用一個來表示

保持一定數量的長連接

一個長連接只能響應一個請求

與第一個server保留32個，意味着一次性可以發出32個長連接，也不用建立新連接

與第二個server保留32個長連接，也是一樣，不用建立新連接，一共就是64個

所以連接一直在，也不用三次握手和四次斷開

這時候，worker在nginx端佔用的端口數量也就不變了

因此，32個長連接連接着，大量的請求就可以通過這些長連接發送

不用每個連接都建立一個端口

可以極大地節約端口

keepalive connections 表示保留的空閒長連接有多少個

假如沒人用這些長連接了，一直建立這些長連接也是對後端服務器的壓力

解決辦法就是超時關閉，最少保持一個關閉

後面訪問就直接可以用了，不用再去建立長連接

keepalive connections ，因此將該設置數字調高一些，可以一定意義上提升服務器性能