多個tomcat要一起協同工作有幾種辦法,可以考慮的方案有以下幾個:
1. 使用tomcat自帶的cluster方式,多個tomcat間自動實時複製session信息,配置起來很簡單。但這個方案的效率比較低,在大併發下表現並不好。
2. 利用nginx的基於訪問ip的hash路由策略,保證訪問的ip始終被路由到同一個tomcat上,這個配置更簡單。但如果應用是某一個局域網大量用戶同時登錄,這樣負載均衡就沒什麼作用了。
3. 利用memcached把多個tomcat的session集中管理,前端在利用nginx負載均衡和動靜態資源分離,在兼顧系統水平擴展的同時又能保證較高的性能。
這裏選擇方案3。
二、環境
系統:Linux
軟件:nginx-1.2.7,memcached-1.4.15,tomcat7。
依賴:libevent-2.0.21-stable(memcached),
asm-3.2.jar
minlog-1.2.jar
reflectasm-1.0.jar
kryo-1.03.jar
kryo-serializers-0.9.jar(如果在官方下載的有問題,我就遇到缺少了DateSerializer類,請自己去git上打包:https://github.com/magro/kryo-serializers)
memcached-session-manager-1.6.3.jar
memcached-session-manager-tc7-1.6.3.jar
memcached-session-manager-tc6-1.6.3.jar
msm-kryo-serializer-1.6.3.jar
spymemcached-2.8.4.jar
三、搭建
1.Memcached(官網:http://memcached.org/):
tar zxvf libevent-2.0.21-stable.tar.gz
cd libevent-2.0.21-stable
./configure --prefix=/desp/local/libevent-2.0.21
make
make install
-------------------------------------------------------------------
tar zxvf memcached-1.4.15.tar.gz
cd memcached-1.4.15
./configure --prefix=/desp/local/memcached-1.4.15 --with-libevent=/desp/local/libevent-2.0.21/
make
make install
2.Ngnix(如想安裝其他模塊,請參考官網:http://nginx.org/cn/):
tar zxvf nginx-1.2.7.tar.gz
cd nginx-1.2.7
./configure --prefix=/desp/ nginx/
make
make install
修改nginx.conf(http://nginx.org/cn/docs/)
#user nobody;
worker_processes 16;
pid sbin/nginx.pid;
worker_rlimit_nofile 51200;
events {
use epoll;
worker_connections 65535;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
client_header_buffer_size 32k;
large_client_header_buffers 4 32k;
client_max_body_size 20m;
client_body_buffer_size 128k;
sendfile on;
tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
gzip on;
gzip_types text/javascript text/plain text/css application/xml application/x-javascript;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
proxy_connect_timeout 300;
proxy_send_timeout 300;
proxy_read_timeout 300;
proxy_buffer_size 16k;
proxy_buffers 4 32k;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Connection Close;
server_names_hash_max_size 1024;
server_names_hash_bucket_size 1024;
# Default cache parameters for use by virtual hosts
# Set the cache path to tmpfs mounted disk, and the zone name
# Set the maximum size of the on disk cache to less than the tmpfs file system size
proxy_cache_path ./cache levels=1:2 keys_zone=pscms:100m max_size=800m;
proxy_temp_path ./proxy;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
#配置後端服務器信息
upstream web_server {
#ip_hash;
server 172.168.1.222:8080 max_fails=3 fail_timeout=30s;
server 172.168.1.222:8081 max_fails=3 fail_timeout=30s;
}
server {
listen 8888; ## listen for ipv4
#listen [::]:80 default ipv6only=on; ## listen for ipv6
server_name 172.168.1.221;
charset utf-8;
access_log logs/host.access.log main;
#access_log off;
location ~ .*\.(jsp|action)?$ {
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://web_server;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location ~ .*\.(gif|jpg|jpeg|png|bmp|swf|js|css)$ {
#如果後端的服務器返回502、504、執行超時等錯誤,自動將請求轉發到upstream負載均衡池中的另一臺服務器,實現故障轉移。
proxy_next_upstream http_502 http_504 error timeout invalid_header;
proxy_cache pscms; #進行緩存,使用Web緩存區cache_one
proxy_cache_valid 200 304 1h; #對不同的HTTP狀態碼設置不同的緩存時間
proxy_cache_valid 301 302 5m;
proxy_cache_valid any 1m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Accept-Encoding ""; #(或是後臺服務器關閉gzip),這樣這臺機器纔不會緩存被壓縮的文件,造成亂碼
proxy_ignore_headers "Cache-Control" "Expires"; #這段配置加上後,proxy_cache就能支持後臺設定的expires。
proxy_pass http://web_server;
expires 15m;
}
location / {
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://web_server;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}
3.tomcat
把asm-3.2.jar,minlog-1.2.jar,reflectasm-1.0.jar,kryo-1.03.jar,kryo-serializers-0.9.jar,memcached-session-manager-1.6.3.jar,memcached-session-manager-tc7-1.6.3.jar(有6、7兩個jar分別對應tomcat6、7),msm-kryo-serializer-1.6.3.jar,spymemcached-2.8.4.jar放到tomcat/lib/下.
修改context.xml或者server.xml:
<Engine name="Catalina" defaultHost="localhost" jvmRoute="jvm1">
jvmroute就是給memcached的標識。
<Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
memcachedNodes="n1:172.168.1.221:11211"
requestUriIgnorePattern=".*\.(png|gif|jpg|css|js|ico|jpeg|htm|html)$"
sessionBackupAsync="false"
sessionBackupTimeout="1800000"
copyCollectionsForSerialization="false"
transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
/>
更多選項請查看:https://github.com/magro/memcached-session-manager/wiki/SetupAndConfiguration
此項目地址:https://github.com/magro/memcached-session-manager
四:原理
原文:http://xylonwang.iteye.com/blog/1269704,http://miwucc.iteye.com/blog/1329729
MSM(memcached-session-manager) 支持tomcat6 和tomcat7 ,利用 Value(Tomcat 閥)對Request進行跟蹤。Request請求到來時,從memcached加載session,Request請求結束時,將tomcat session更新至memcached,以達到session共享之目的, 支持 sticky 和 non-sticky 模式。
1.Sticky 模式:
tomcat session 爲主session,memcached 爲備session。Request請求到來時, 從memcached加載備 session 到 tomcat (僅當tomcat jvmroute發生變化時,否則直接取tomcat session);Request請求結束時,將tomcat session更新至memcached,以達到主備同步之目的。
2.Non-Sticky模式:
tomcat session 爲 中轉session, memcached1 爲主 session,memcached 2 爲備session。Request請求到來時,從memcached 2加載備 session 到 tomcat,(當 容器 中還是沒有session 則從memcached1加載主 session 到 tomcat, 這種情況是隻有一個memcached節點,或者有memcached1 出錯時),Request請求結束時,將tomcat session更新至 主memcached1和備memcached2,並且清除tomcat session 。以達到主備同步之目的。