Nginx反代多臺tomcat，並實現session共享

實驗介紹：當一臺服務器無法響應過多的請求時，此時的方案再增加一臺服務器，以減少服務器的壓力。也可以防止服務器宕機後，無法訪問網站。

實驗環境：centos 6.3 64bit，tomcat 7，nginx 1.2.6

一，給nginx打補丁

#wget https://nodeload.github.com/yaoweibin/nginx_upstream_check_module/zip/master

#wget http://nginx-upstream-jvm-route.googlecode.com/files/nginx-upstream-jvm-route-0.1.tar.gz

[root@test1 nginx-1.2.6]# patch -p0 < /root/nginx_upstream_jvm_route/jvm_route.patch

[root@test1 nginx-1.2.6]# patch -p1< /root/nginx_upstream_check_module-master/check_1.2.6+.patch

#yum -y install zlib zlib-devel openssl openssl-devel pcre pcre-devel

[root@test1 nginx-1.2.6]# ./configure --prefix=/usr/local/nginx --with-http_ssl_module --with-http_flv_module --with-http_stub_status_module --with-http_gzip_static_module --add-module=/root/nginx_upstream_jvm_route/ --add-module=/root/nginx_upstream_check_module-master[root@test1 nginx-1.2.6]# make && make install

不打補丁會提示下面錯誤

[root@test1 nginx-1.2.6]# make

make -f objs/Makefile

make[1]: Entering directory `/root/nginx-1.2.6'

gcc -c -pipe -O -W -Wall -Wpointer-arith -Wno-unused-parameter -Werror -g -I src/core -I src/event -I src/event/modules -I src/os/unix -I objs -I src/http -I src/http/modules -I src/mail \

-o objs/addon/nginx_upstream_jvm_route/ngx_http_upstream_jvm_route_module.o \

/root/nginx_upstream_jvm_route/ngx_http_upstream_jvm_route_module.c

/root/nginx_upstream_jvm_route/ngx_http_upstream_jvm_route_module.c: In function ?.gx_http_upstream_jvm_route_get_socket?.

/root/nginx_upstream_jvm_route/ngx_http_upstream_jvm_route_module.c:191: error: ?.gx_http_upstream_srv_conf_t?.has no member named ?.everse?

/root/nginx_upstream_jvm_route/ngx_http_upstream_jvm_route_module.c:192: error: ?.gx_http_upstream_server_t?.has no member named ?.run_id?

/root/nginx_upstream_jvm_route/ngx_http_upstream_jvm_route_module.c:193: error: ?.gx_http_upstream_server_t?.has no member named ?.run_id?

/root/nginx_upstream_jvm_route/ngx_http_upstream_jvm_route_module.c:198: error: ?.gx_http_upstream_server_t?.has no member named ?.run_id?

二，配置nginx.conf

#vim nginx.conf

#user nobody;

worker_processes 8;

#error_log logs/error.log;

#error_log logs/error.log notice;

#error_log logs/error.log info;

#pid logs/nginx.pid;

events {

use epoll;

worker_connections 1024;

}

http {

include mime.types;

default_type application/octet-stream;

sendfile on;

tcp_nopush on;

keepalive_timeout 60;

tcp_nodelay on;

gzip on;

gzip_min_length 1k;

gzip_buffers 4 16k;

gzip_http_version 1.0;

gzip_comp_level 2;

gzip_types text/plain application/x-javascript text/css application/xml;

gzip_vary on;

server_names_hash_bucket_size 128;

client_header_buffer_size 32k;

large_client_header_buffers 4 32k;

client_max_body_size 10m;

client_body_buffer_size 128k;

proxy_connect_timeout 600;

proxy_read_timeout 600;

proxy_send_timeout 600;

proxy_buffer_size 8k;

proxy_buffers 4 32k;

proxy_busy_buffers_size 64k;

proxy_temp_file_write_size 64k;

upstream tomcat {

ip_hash;

server 192.168.1.22:8080;

server 192.168.1.23:8080;

check interval=3000 rise=2 fall=5 timeout=1000;

}

server {

listen 80;

server_name localhost;

index index.jsp index.action;

# root /usr/local/tomcat/app1/apps/fis;

location ~ .*\.(jsp|action|js)$ {

proxy_pass http://tomcat;

proxy_redirect off;

proxy_set_header Host $http_host;

proxy_set_header X-Real-IP $remote_addr;

proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

chunked_transfer_encoding off;

}

location /status {

check_status;

access_log off;

}

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root html;

}

註釋：

在nginx配置文件中，採用了ip_hash調度算法，ip_hash工作模式，對於每個請求按訪問ip的hash結果分配，這樣來自同一個ip的訪問用戶固定訪問一個後臺的服務器，這個能有效解決動態網頁session共享問題。當然還有其他工作模式。

check interval=3000 rise=2 fall=5 timeout=1000;這個是後臺的健康檢測

三，建立nginx腳本：

Vim /etc/init.d/nginx

#!/bin/sh

# nginx - this script starts and stops the nginx daemon

# chkconfig: - 85 15

# description: Nginx is an HTTP(S) server, HTTP(S) reverse \

# proxy and IMAP/POP3 proxy server

# processname: nginx

# config: /etc/nginx/nginx.conf

# config: /etc/sysconfig/nginx

# pidfile: /var/run/nginx.pid

# Source function library.

. /etc/rc.d/init.d/functions

# Source networking configuration.

. /etc/sysconfig/network

# Check that networking is up.

[ "$NETWORKING" = "no" ] && exit 0

nginx="/usr/local/nginx/sbin/nginx"

prog=$(basename $nginx)

NGINX_CONF_FILE="/usr/local/nginx/conf/nginx.conf"

[ -f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx

lockfile=/var/lock/subsys/nginx

make_dirs() {

# make required directories

user=`nginx -V 2>&1 | grep "configure arguments:" | sed 's/[^*]*--user=$[^ ]*$.*/\1/g' -`

options=`$nginx -V 2>&1 | grep 'configure arguments:'`

for opt in $options; do

if [ `echo $opt | grep '.*-temp-path'` ]; then

value=`echo $opt | cut -d "=" -f 2`

if [ ! -d "$value" ]; then

# echo "creating" $value

mkdir -p $value && chown -R $user $value

done

}

start() {

[ -x $nginx ] || exit 5

[ -f $NGINX_CONF_FILE ] || exit 6

make_dirs

echo -n $"Starting $prog: "

daemon $nginx -c $NGINX_CONF_FILE

retval=$?

echo

[ $retval -eq 0 ] && touch $lockfile

return $retval

}

stop() {

echo -n $"Stopping $prog: "

killproc $prog -QUIT

retval=$?

echo

[ $retval -eq 0 ] && rm -f $lockfile

return $retval

}

restart() {

configtest || return $?

stop

sleep 1

start

}

reload() {

configtest || return $?

echo -n $"Reloading $prog: "

killproc $nginx -HUP

RETVAL=$?

echo

}

force_reload() {

restart

}

configtest() {

$nginx -t -c $NGINX_CONF_FILE

}

rh_status() {

status $prog

}

rh_status_q() {

rh_status >/dev/null 2>&1

}

case "$1" in

start)

rh_status_q && exit 0

;;

stop)

rh_status_q || exit 0

;;

restart|configtest)

;;

reload)

rh_status_q || exit 7

;;

force-reload)

force_reload

;;

status)

rh_status

;;

condrestart|try-restart)

rh_status_q || exit 0

;;

echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}"

exit 2

esac

打開nginx默認網頁測試nginx是否安裝成功

安裝tomcatA

四，配置tomcatA

[root@test1 ~]# rpm -ivh jdk-7u9-linux-x64.rpm

[root@test1 ~]# vim /etc/profile 在export做下列修改和最一行修改ulimit

JAVA_HOME=/usr/java/jdk1.7.0_09/

CLASS_PATH=$JAVA_HOME/lib:$JAVA_HOME/jre/lib

PATH=$PATH:$JAVA_HOME:/bin

CATALINA_HOME=/usr/local/tomcat

export JAVA_HOME CATALINA_HOME

export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL HISTTIMEFORMAT

unset i

unset pathmunge

ulimit -SHn 65535

[root@test1 ~]# . /etc/profile

[root@test1 ~]# java -version

java version "1.7.0_09"

Java(TM) SE Runtime Environment (build 1.7.0_09-b05)

Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode)

[root@test1 ~]# tar xf apache-tomcat-7.0.32.tar.gz -C /usr/local/

[root@test1 ~]# cd /usr/local/

[root@test1 local]# ln -s apache-tomcat-7.0.32 tomcat

[root@test1 local]# cd tomcat/

[root@test1 tomcat]#mkdir webapps/test/WEB-INF -pv

[root@test1 tomcat]# cd webapps/test

[root@test1 test]# vim test.jsp

<%@ page language="java" %>

<html>

<head><title>TomcatA</title></head>

<body>

<h1><fontcolor="red">TomcatA </font></h1>

<tablealign="centre"border="1">

<tr>

<td>Session ID</td>

</tr>

<tr>

<td>Created on</td>

</tr>

</table>

</body>

</html>

[root@test1 test]# vim WEB-INF/web.xml 不能修改conf/web.xml，修改conf/web.xml session並不會複製。只能修改對應項目下的web.xml。

<web-app xmlns="http://java.sun.com/xml/ns/j2ee"

xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"

xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd"

version="2.5">

<display-name>TomcatDemo</display-name>

<distributable/> 添加一行

</web-app>

註釋：添加<distributable/>屬性，表示應用程序爲分佈式

五，配置tomcat集羣:作用讓tomcat之間同步session

[root@test1 tomcat]# vim conf/server.xml

<!--For clustering, please take a look at documentation at:

/docs/cluster-howto.html (simple how to)

/docs/config/cluster.html (reference documentation) -->

<!--

-->

<Manager className="org.apache.catalina.ha.session.DeltaManager" expireSessionsOnShutdown="false"

notifyListenersOnReplication="true"/>

<MemberShip className="org.apache.catalina.tribes.membership.McastService"

bind="192.168.1.22"

address="228.0.0.4"

port="45564"

frequency="500"

droptTime="3000"/>

<Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"

address="192.168.1.22"

port="4000"

autoBind="100"

selectorTimeout="5000"

maxThreads="6"/>

</Sender>

</Channel>

</Cluster>

對192.168.1.23配置tomcat做同樣的配置，只需要將test.jsp中tomcatA改成tomcatB

將server.xml中IP改成192.168.1.23，tomcatA改變tomcatB。

啓動tomcat並做如下測試：兩臺tomcat目前正常。

打開test.jsp測試頁，現在訪問的tomcatA，繼續用相同瀏覽器打開，訪問的還是tomcatA，並且會話ID是不會變的

將tomcatA的服務停了，驗證會不會自動將會話複製到tomcatB,

#bin/catalina.sh stop

繼續刷新，此時tomcatA變成tomcatB,會話ID並不變化，結果證明成功。

[root@~]$ grep -i session logs/catalina.out 查看日誌

信息: Manager [telcom.yeezhao.com#], requesting session state from org.apache.catalina.tribes.membership.MemberImpl[tcp://{222, 200, 185, 27}:4000,{222, 200, 185, 27},4000, alive=290104, securePort=-1, UDP Port=-1, id={124 -108 -44 67 40 110 73 -36 -82 -88 -1 88 67 -91 46 43 }, payload={}, command={}, domain={}, ]. This operation will timeout if no session state has been received within 60 seconds.

2013-8-29 14:45:25 org.apache.catalina.ha.session.DeltaManager waitForSendAllSessions

信息: Manager [telcom.yeezhao.com#]; session state send at 8/29/13 2:45 PM received in 105 ms.

在生產環境中驗證，先查看訪問的是哪臺tomcat，並登錄，將訪問tomcat 停了，刷新網頁查看會不會要求重新登錄，如果不會則session複製成功。

Nginx反代多臺tomcat，並實現session共享

SQL優化-20231016

Centos 6.3 x86_64 配置rsync+inotify

Centos 6.3 x86_64配置rsync

Centos 6.3 x86_64安裝與配置bacula

nginx+tomcat+ngx_cache_purge配置

我的友情鏈接

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結