Nginx+Memcached+Tomcat集羣配置實踐（Sticky Session）

原創 千与 2018-08-30 20:13
準備工作

創建一個簡單的web應用，名爲session。其中有兩個頁面，分別如下所示：
  • 頁面login.jsp
<%@ page language="java" contentType="text/html; charset=UTF-8"
     pageEncoding="UTF-8"%>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>登錄頁面</title>
<meta http-equiv="pragma" content="no-cache">
<meta http-equiv="cache-control" content="no-cache">
<meta http-equiv="expires" content="0">
<meta http-equiv="keywords" content="keyword1,keyword2,keyword3">
<meta http-equiv="description" content="This is my page">
</head>
<body>
     <table bgcolor="#F8C3C3" border="1" borderColor="#FF0000"
          align="center" width="260" height="160">
          <tr>
               <td height="15" colspan="2" align="center"
                    style="background-color: #000000; font-size: 28px; color: #FFFF00"><b>用戶登錄</b></td>
          </tr>
          <form method="post" action="login">
               <tr>
                    <td width="80" align="center"><b>用戶名</b></td>
                    <td><input type="text" name="userName" value="" /></td>
               </tr>
               <tr>
                    <td align="center"><b>密 碼</b></td>
                    <td><input type="password" name="password" value="" /></td>
               </tr>
               <tr>
                    <td colspan="2" align="center" style="background-color: #000000;">
                         <input type="submit" value="登 錄" />  <input type="reset"
                         value="重 置" />
                    </td>
               </tr>
          </form>
     </table>
</body>
</html>
  • 登錄成功頁面success.jsp
<%@ page language="java" import="java.util.*" pageEncoding="UTF-8"%>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>登錄成功頁面</title>
<meta http-equiv="pragma" content="no-cache">
<meta http-equiv="cache-control" content="no-cache">
<meta http-equiv="expires" content="0">
<meta http-equiv="keywords" content="keyword1,keyword2,keyword3">
<meta http-equiv="description" content="This is my page">
</head>
<%
     String userName = (String) request.getSession().getAttribute("userName");
%>
<body bgcolor="#000000">
     <table width="500" height="200" align="center" border="1"
          style="background-color: #000000;">
          <tr>
               <td align="center"
                    style="background-color: #F8C3C3; color: #00FF00; font-weight: bold; font-size: 30px">
                    當前登錄用戶：<%=userName %>
               </td>
          </tr>
     </table>
</body>
</html>
還有一個Servlet，負責請求的轉發，並設置session數據，如下所示：
package org.shirdrn.cluster.tomcat.servlet;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;

public class LoginSevlet extends HttpServlet {
     private static final long serialVersionUID = 1L;
     private static final Log LOG = LogFactory.getLog(LoginSevlet.class);
    private static Map<String, String> accountDB = new HashMap<String, String>();
    static {
         accountDB.put("shirdrn", "123456");
         accountDB.put("admin", "999999");
         accountDB.put("dev", "000888");
         accountDB.put("007", "007007");
    }
   
    public LoginSevlet() {
        super();
    }

     protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
          this.doPost(request, response);
     }

     protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
          String userName = request.getParameter("userName");
          String password = request.getParameter("password");
          LOG.info("Raw input:userName=" + userName + ",password=" + password);
         
          if(userName!=null && password!=null
                    && accountDB.containsKey(userName)
                    && accountDB.get(userName).equals(password)) {
               LOG.info("Login;status=SUCCESS");
               request.getSession().setAttribute("userName", userName);
               request.getRequestDispatcher("success.jsp").forward(request, response);              
          } else {
               LOG.info("Login;status=FAIL");
               request.getRequestDispatcher("login.jsp").forward(request, response);
          }
     }

}
這裏模擬了一個賬號數據庫，對用戶的登錄請求進行處理。
最後打成WAR包，部署到Tomcat集羣中的每個結點上。
Memcached安裝、配置、啓動

在Ubuntu系統下，如果沒有安裝Memcached，可以執行如下命令進行安裝：
sudo apt-get install memcached
然後，啓動Memcached服務進程，這裏，我啓動了兩個服務進程，分別監聽端口11211、11311，啓動命令如下所示：
sudo /usr/bin/memcached -m 64 -d -u memcache -p 11211 -l 192.168.1.104 -c 5000 -P /tmp/n1-memcached.pid
sudo /usr/bin/memcached -m 64 -d -u memcache -p 11311 -l 192.168.1.104 -c 5000 -P /tmp/n2-memcached.pid
查詢Memcached服務狀態：
shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ ps -ef | grep memcached
memcache  1578     1  0 Jan25 ?        00:00:00 /usr/bin/memcached -m 64 -p 11211 -u memcache -l 127.0.0.1
memcache  3300     1  0 Jan25 ?        00:00:00 /usr/bin/memcached -m 64 -d -u memcache -p 11211 -l 192.168.1.104 -c 5000 -P /tmp/n1-memcached.pid
memcache  3308     1  0 00:00 ?        00:00:00 /usr/bin/memcached -m 64 -d -u memcache -p 11311 -l 192.168.1.104 -c 5000 -P /tmp/n2-memcached.pid
shirdrn   3318  3095  0 00:00 pts/5    00:00:00 grep --color=auto memcached

Tomcat集羣配置

Tomcat集羣配置，集羣中各個結點通過共享存儲在緩存Memcached中session來實現session的共享：如果有一臺機器上的Tomcat服務停掉了，對於其他對等服務器上的session數據仍然可以從Memcached緩存中讀取，從而不會發生session丟失的問題。
對於滿足這種配置的實現方案，可以在https://code.google.com/p/memcached-session-manager和https://spymemcached.googlecode.com上下載相關的jar文件。因爲基於不同的序列化方案，可以有多種配置方法，下面是選擇Javolution序列化框架，需要提供如下庫文件：
下載上述jar文件，拷貝到$CATALINA_HOME/lib目錄下，然後需要配置$CATALINA_HOME/conf/server.xml文件。
Tomcat集羣中，每個結點的$CATALINA_HOME/conf/server.xml基本配置都是相同的，不同配置內容分別如下：
  • 如果在同一臺機器上，要保證各個服務端口不相沖突
  • <Manager>元素中failoverNodes屬性值不同
例如，我們在同一臺機器上配置了兩個Tomcat服務器實例，所在目錄分別爲tomcat-1和tomcat-2，對應的conf/server.xml配置內容如下：
  • tomcat-1/conf/server.xml
<?xml version='1.0' encoding='utf-8'?>

<Server port="8085" shutdown="SHUTDOWN">
     <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
     <Listener className="org.apache.catalina.core.JasperListener" />
     <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
     <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
     <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

     <GlobalNamingResources>
          <Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase"
               description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
               pathname="conf/tomcat-users.xml" />
     </GlobalNamingResources>

     <Service name="Catalina">
          <Connector port="8088" protocol="HTTP/1.1" connectionTimeout="20000"
               redirectPort="8443" />
          <Connector port="8089" protocol="AJP/1.3" redirectPort="8443" />
          <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">

               <Realm className="org.apache.catalina.realm.LockOutRealm">
                    <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                         resourceName="UserDatabase" />
               </Realm>

               <Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true" deployOnStartup="true">
                    <Context docBase="/home/shirdrn/servers/cluster/nginx_tomcat_memcached/webapps/session.war" path="/session" reloadable="true">
                         <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
                                            memcachedNodes="n1:192.168.1.104:11211,n2:192.168.1.104:11311"
                                            failoverNodes="n1"
                                            requestUriIgnorePattern=".*\.(png|gif|jpg|css|js)$"
                                            sessionBackupAsync="false"
                                            sessionBackupTimeout="100"
                             transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"
                             copyCollectionsForSerialization="false" />
                    </Context>
                    <Valve className="org.apache.catalina.valves.AccessLogValve"
                         directory="logs" prefix="localhost_access_log." suffix=".txt"
                         pattern="%h %l %u %t "%r" %s %b" />
               </Host>

          </Engine>
     </Service>
</Server>
這裏面，failover的Memcached結點是n1，也就是說，如果tomcat-1和n1在同一臺機器上的話，session數據會優先複製到存儲到Memcached結點n2，這樣即使n1所在的結點宕機了，n2所在結點仍然存儲了之前的session數據。
  • tomcat-2/conf/server.xml
<?xml version='1.0' encoding='utf-8'?>

<Server port="9085" shutdown="SHUTDOWN">
     <Listener className="org.apache.catalina.core.AprLifecycleListener" SSLEngine="on" />
     <Listener className="org.apache.catalina.core.JasperListener" />
     <Listener className="org.apache.catalina.core.JreMemoryLeakPreventionListener" />
     <Listener className="org.apache.catalina.mbeans.GlobalResourcesLifecycleListener" />
     <Listener className="org.apache.catalina.core.ThreadLocalLeakPreventionListener" />

     <GlobalNamingResources>
          <Resource name="UserDatabase" auth="Container" type="org.apache.catalina.UserDatabase"
               description="User database that can be updated and saved" factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
               pathname="conf/tomcat-users.xml" />
     </GlobalNamingResources>

     <Service name="Catalina">
          <Connector port="9088" protocol="HTTP/1.1" connectionTimeout="20000"
               redirectPort="9443" />
          <Connector port="9089" protocol="AJP/1.3" redirectPort="9443" />
          <Engine name="Catalina" defaultHost="localhost" jvmRoute="tomcat1">

               <Realm className="org.apache.catalina.realm.LockOutRealm">
                    <Realm className="org.apache.catalina.realm.UserDatabaseRealm"
                         resourceName="UserDatabase" />
               </Realm>

               <Host name="localhost" appBase="webapps" unpackWARs="true" autoDeploy="true" deployOnStartup="true">
                    <Context docBase="/home/shirdrn/servers/cluster/nginx_tomcat_memcached/webapps/session.war" path="/session" reloadable="true">
                         <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
                                            memcachedNodes="n1:192.168.1.104:11211,n2:192.168.1.104:11311"
                                            failoverNodes="n2"
                                            requestUriIgnorePattern=".*\.(png|gif|jpg|css|js)$"
                                            sessionBackupAsync="false"
                                            sessionBackupTimeout="100"
                             transcoderFactoryClass="de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory"
                             copyCollectionsForSerialization="false" />
                    </Context>
                    <Valve className="org.apache.catalina.valves.AccessLogValve"
                         directory="logs" prefix="localhost_access_log." suffix=".txt"
                         pattern="%h %l %u %t "%r" %s %b" />
               </Host>

          </Engine>
     </Service>
</Server>
和上面類似，failover的Memcached結點是n2，也就是說，如果tomcat-2和n2在同一臺機器上的話，Session數據會優先複製到存儲到Memcached結點n1，這樣即使n2所在的結點宕機了，n1所在結點仍然存儲了之前的Session數據。

上面的配置實現了Tomcat集羣中，通過Memcached實現了Sticky Session（粘性Session），主要是通過配置failover結點來達到目的的。如果採用Non-sticky Session方式，就不需要考慮failover的問題，詳細配置擴參考文檔（https://code.google.com/p/memcached-session-manager/wiki/SetupAndConfiguration），下面截取一個片段：

The following example shows a configuration for non-sticky sessions. In this case there's no need for failoverNodes, as sessions are served by all tomcats round-robin and they're not bound to a single tomcat. For non-sticky sessions the configuration (for both/all tomcats) would look like this:

<Context>
  ...
  <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
    memcachedNodes="n1:host1.yourdomain.com:11211,n2:host2.yourdomain.com:11211"
    sticky="false"
    sessionBackupAsync="false"
    lockingMode="uriPattern:/path1|/path2"
    requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
    transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
    />
</Context>
分別啓動tomcat-1和tomcat-2，如下所示：
shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tomcat-1/bin/catalina.sh start
Using CATALINA_BASE:   /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1
Using CATALINA_HOME:   /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1
Using CATALINA_TMPDIR: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/temp
Using JRE_HOME:        /home/hadoop/installation/jdk1.6.0_30
Using CLASSPATH:       /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/bin/bootstrap.jar:/home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/bin/tomcat-juli.jar

shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tomcat-2/bin/catalina.sh start
Using CATALINA_BASE:   /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2
Using CATALINA_HOME:   /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2
Using CATALINA_TMPDIR: /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/temp
Using JRE_HOME:        /home/hadoop/installation/jdk1.6.0_30
Using CLASSPATH:       /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/bin/bootstrap.jar:/home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/bin/tomcat-juli.jar
可以查看兩個tomcat的啓動日誌，對比一下：
  • tomcat-1的日誌
shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tail -100f tomcat-1/logs/catalina.out
Jan 26, 2013 12:03:25 AM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
Jan 26, 2013 12:03:26 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8088"]
Jan 26, 2013 12:03:26 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-8089"]
Jan 26, 2013 12:03:26 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 1743 ms
Jan 26, 2013 12:03:26 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Jan 26, 2013 12:03:26 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.26
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO: MemcachedSessionService starts initialization... (configured nodes definition n1:192.168.1.104:11211,n2:192.168.1.104:11311, failover nodes n1)
2013-01-26 00:03:27.441 INFO net.spy.memcached.MemcachedConnection:  Added {QA sa=/192.168.1.104:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-01-26 00:03:27.442 INFO net.spy.memcached.MemcachedConnection:  Added {QA sa=/192.168.1.104:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.RequestTrackingHostValve <init>
INFO: Setting ignorePattern to .*\.(png|gif|jpg|css|js)$
2013-01-26 00:03:27.459 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@5b0668
2013-01-26 00:03:27.460 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@1d5a0
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode
INFO: Setting lockingMode to null
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory
INFO: Creating transcoder factory de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory
Jan 26, 2013 12:03:27 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids [n2] and failover node ids [n1]
Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/host-manager
Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/manager
Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/docs
Jan 26, 2013 12:03:27 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/examples
Jan 26, 2013 12:03:28 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-1/webapps/ROOT
Jan 26, 2013 12:03:28 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8088"]
Jan 26, 2013 12:03:28 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8089"]
Jan 26, 2013 12:03:28 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 1487 ms
  • tomcat-2的日誌
shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ tail -100f tomcat-2/logs/catalina.out
Jan 26, 2013 12:04:34 AM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /home/hadoop/installation/jdk1.6.0_30/jre/lib/i386/client:/home/hadoop/installation/jdk1.6.0_30/jre/lib/i386:/home/hadoop/installation/jdk1.6.0_30/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
Jan 26, 2013 12:04:34 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-9088"]
Jan 26, 2013 12:04:34 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-9089"]
Jan 26, 2013 12:04:34 AM org.apache.catalina.startup.Catalina load
INFO: Initialization processed in 742 ms
Jan 26, 2013 12:04:34 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Catalina
Jan 26, 2013 12:04:34 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.26
Jan 26, 2013 12:05:07 AM org.apache.catalina.util.SessionIdGenerator createSecureRandom
INFO: Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [33,094] milliseconds.
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO: MemcachedSessionService starts initialization... (configured nodes definition n1:192.168.1.104:11211,n2:192.168.1.104:11311, failover nodes n2)
2013-01-26 00:05:07.873 INFO net.spy.memcached.MemcachedConnection:  Added {QA sa=/192.168.1.104:11211, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
2013-01-26 00:05:07.876 INFO net.spy.memcached.MemcachedConnection:  Added {QA sa=/192.168.1.104:11311, #Rops=0, #Wops=0, #iq=0, topRop=null, topWop=null, toWrite=0, interested=0} to connect queue
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.RequestTrackingHostValve <init>
INFO: Setting ignorePattern to .*\.(png|gif|jpg|css|js)$
2013-01-26 00:05:07.896 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@701a27
2013-01-26 00:05:07.897 INFO net.spy.memcached.MemcachedConnection:  Connection state changed for sun.nio.ch.SelectionKeyImpl@1d5a0
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService setLockingMode
INFO: Setting lockingMode to null
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService createTranscoderFactory
INFO: Creating transcoder factory de.javakaffee.web.msm.serializer.javolution.JavolutionTranscoderFactory
Jan 26, 2013 12:05:07 AM de.javakaffee.web.msm.MemcachedSessionService startInternal
INFO: MemcachedSessionService finished initialization, sticky true, operation timeout 1000, with node ids [n1] and failover node ids [n2]
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/host-manager
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/manager
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/docs
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/examples
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /home/shirdrn/servers/cluster/nginx_tomcat_memcached/tomcat-2/webapps/ROOT
Jan 26, 2013 12:05:08 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-9088"]
Jan 26, 2013 12:05:08 AM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-9089"]
Jan 26, 2013 12:05:08 AM org.apache.catalina.startup.Catalina start
INFO: Server startup in 34163 ms
已經成功啓動了。

Nginx安裝、配置、啓動

在Ubuntu系統下，如果沒有安裝Nginx，可以執行如下命令進行安裝：
sudo apt-get install nginx
默認情況下，Nginx使用默認的配置文件/etc/nginx/nginx.conf，該配置文件直接包含了虛擬目錄配置文件/etc/nginx/sites-available/default，修改該文件的內容爲，如下所示：
# You may add here your
# server {
#     ...
# }
# statements for each of your virtual hosts to this file

##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# http://wiki.nginx.org/Pitfalls
# http://wiki.nginx.org/QuickStart
# http://wiki.nginx.org/Configuration
#
# Generally, you will want to move this file somewhere, and start with a clean
# file but keep this around for reference. Or just disable in sites-enabled.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##



upstream dev.shirdrn.org {
     server 192.168.1.104:8088 weight=1;
     server 192.168.1.104:9088 weight=1;
}


server {
     #listen   80; ## listen for ipv4; this line is default and implied
     #listen   [::]:80 default ipv6only=on; ## listen for ipv6

     root /usr/share/nginx/www/session;  # session是我們配置的虛擬目錄，實際直接指向Tomcat下名爲session的web應用
     index index.html index.htm;

     # Make site accessible from http://localhost/
#######     server_name localhost;
     server_name dev.shirdrn.org; # Nginx服務所在主機
     charset utf-8;

     location / {
          proxy_pass     http://dev.shirdrn.org; #直接代理tomcat集羣
          proxy_set_header  X-Real-IP  $remote_addr;
          client_max_body_size  100m;



          # First attempt to serve request as file, then
          # as directory, then fall back to index.html
#######          try_files $uri $uri/ /index.html;
          # Uncomment to enable naxsi on this location
          # include /etc/nginx/naxsi.rules
     }

     location ~ ^/(WEB-INF)/ { # 禁止訪問Tomcat下web應用的WEB-INF目錄下的資源
          deny all;
     }

     location /doc/ {
          alias /usr/share/doc/;
          autoindex on;
          allow 127.0.0.1;
          deny all;
     }

     # Only for nginx-naxsi : process denied requests
     #location /RequestDenied {
          # For example, return an error code
          #return 418;
     #}

     #error_page 404 /404.html;

     # redirect server error pages to the static page /50x.html
     #
     #error_page 500 502 503 504 /50x.html;
     #location = /50x.html {
     #     root /usr/share/nginx/www;
     #}

     # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
     #
     #location ~ \.php$ {
     #     fastcgi_split_path_info ^(.+\.php)(/.+)$;
     #     # NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
     #
     #     # With php5-cgi alone:
     #     fastcgi_pass 127.0.0.1:9000;
     #     # With php5-fpm:
     #     fastcgi_pass unix:/var/run/php5-fpm.sock;
     #     fastcgi_index index.php;
     #     include fastcgi_params;
     #}

     # deny access to .htaccess files, if Apache's document root
     # concurs with nginx's one
     #
     #location ~ /\.ht {
     #     deny all;
     #}
}


# another virtual host using mix of IP-, name-, and port-based configuration
#
#server {
#     listen 8000;
#     listen somename:8080;
#     server_name somename alias another.alias;
#     root html;
#     index index.html index.htm;
#
#     location / {
#          try_files $uri $uri/ /index.html;
#     }
#}


# HTTPS server
#
#server {
#     listen 443;
#     server_name localhost;
#
#     root html;
#     index index.html index.htm;
#
#     ssl on;
#     ssl_certificate cert.pem;
#     ssl_certificate_key cert.key;
#
#     ssl_session_timeout 5m;
#
#     ssl_protocols SSLv3 TLSv1;
#     ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv3:+EXP;
#     ssl_prefer_server_ciphers on;
#
#     location / {
#          try_files $uri $uri/ /index.html;
#     }
#}
上面配置中根目錄爲/usr/share/nginx/www，因爲我們使用到了一個虛擬目錄來對應Tomcat下部署的web應用的path，所以需要在/usr/share/nginx/www下面創建這個對應的虛擬目錄：
shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ sudo mkdir /usr/share/nginx/www/session
最後，可以啓動Nginx服務器，執行如下命令：
shirdrn@dev:~$ sudo /etc/init.d/nginx start
查詢Nginx服務啓動狀態：
shirdrn@dev:~/servers/cluster/nginx_tomcat_memcached$ sudo ps -ef | grep nginx
root      3199     1  0 Jan25 ?        00:00:00 nginx: master process /usr/sbin/nginx
www-data  3200  3199  0 Jan25 ?        00:00:00 nginx: worker process
www-data  3201  3199  0 Jan25 ?        00:00:00 nginx: worker process
www-data  3202  3199  0 Jan25 ?        00:00:00 nginx: worker process
www-data  3203  3199  0 Jan25 ?        00:00:00 nginx: worker process
shirdrn   3327  3095  0 00:01 pts/5    00:00:00 grep --color=auto nginx
這是就可以通過Nginx訪問（代理）Web應用首頁了。

集羣驗證

上面配置完成，服務都已經啓動成功，可以直接請求鏈接http://dev.shirdrn.org/session，就會直接訪問部署在Tomcat服務器上的Java Web應用，看到登錄頁面。
使用前面準備的Web應用session中賬號數據進行模擬登錄，就可以查看Memcached緩存的數據：
  • telnet 192.168.1.104 11211
telnet到memcached服務端口，查看緩存統計數據。
shirdrn@dev:~$ telnet 192.168.1.104 11211
Trying 192.168.1.104...
Connected to 192.168.1.104.
Escape character is '^]'.

ERROR
stats
STAT pid 3300
STAT uptime 2148
STAT time 1359131741
STAT version 1.4.13
STAT libevent 2.0.16-stable
STAT pointer_size 32
STAT rusage_user 0.052003
STAT rusage_system 0.084005
STAT curr_connections 7
STAT total_connections 9
STAT connection_structures 8
STAT reserved_fds 20
STAT cmd_get 23
STAT cmd_set 6
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 2
STAT get_misses 21
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 2675
STAT bytes_written 2835
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 262144
STAT hash_is_expanding 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT bytes 858
STAT curr_items 2
STAT total_items 6
STAT evictions 0
STAT reclaimed 0
END
  • telnet 192.168.1.104 11311
shirdrn@dev:~$ telnet 192.168.1.104 11311
Trying 192.168.1.104...
Connected to 192.168.1.104.
Escape character is '^]'.
stats
STAT pid 3308
STAT uptime 2398
STAT time 1359132000
STAT version 1.4.13
STAT libevent 2.0.16-stable
STAT pointer_size 32
STAT rusage_user 0.004000
STAT rusage_system 0.128008
STAT curr_connections 7
STAT total_connections 9
STAT connection_structures 8
STAT reserved_fds 20
STAT cmd_get 1
STAT cmd_set 2
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 1
STAT delete_misses 1
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 644
STAT bytes_written 2109
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 262144
STAT hash_is_expanding 0
STAT expired_unfetched 1
STAT evicted_unfetched 0
STAT bytes 0
STAT curr_items 0
STAT total_items 2
STAT evictions 0
STAT reclaimed 1
END

我們可以停掉某個Tomcat服務，然後觀察Session會話數據的複製過程，能夠保證當前集羣中的會話數據不丟失，另一個結點仍然能提供基於該存在Session的服務。


發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章

Ubuntu下基於Nginx實現Tomcat集羣負載均衡

千与 2020-02-22 14:43:05

Linux下memcached-1.4.10安裝

千与 2018-08-30 20:13:42

Ubuntu系統啓動報錯：The system is running in low-graphics mode

千与 2018-08-30 20:13:40