Tomcat集群session复制,httpd/nginx反代Tomcat集群

一个大型站点都会涉及到动态应用，动态应用都需要做会话保持，常见的会话保持方式就三种，一是session stick，二是session replication，三是session share，对于小型规模的tomcat集群，大多者会采用session replication方式，但阅读官方文档也好，查询大牛博客也罢，发现均有不准确之处，所以亲测成功实现之后得出如下文档，还望高人指点。

实验环境：

操作系统：CentOS 7.2

tomcat版本：tomcat-7.0.54（yum安装方式）

httpd版本：httpd-2.4.6（yum安装方式）

nginx版本：nginx-1.10.2（yum安装方式）

iptables状态：disable

SELinux状态：disable

TomcatA配置过程

[root@TomcatA ~]# ifconfig

eno16777736:flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.0.20 netmask255.255.255.0 broadcast 192.168.0.255

inet6 fe80::20c:29ff:fefe:64f3 prefixlen 64 scopeid0x20<link>

ether 00:0c:29:fe:64:f3 txqueuelen 1000 (Ethernet)

RX packets 1147 bytes 141324(138.0 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 659 bytes 102242 (99.8KiB)

TX errors 0 dropped 0 overruns0 carrier 0 collisions 0

lo:flags=73<UP,LOOPBACK,RUNNING> mtu65536

inet 127.0.0.1 netmask 255.0.0.0

inet6 ::1 prefixlen 128 scopeid 0x10<host>

loop txqueuelen 0 (Local Loopback)

RX packets 0 bytes 0 (0.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 0 bytes 0 (0.0 B)

TX errors 0 dropped 0 overruns0 carrier 0 collisions 0

[root@TomcatA ~]# cat/etc/yum.repos.d/hehe.repo

[cdrom-repo]

name=cdrom-repo

baseurl=file:///cdrom

gpgcheck=0

[aliyun-epel]

name=aliyun-epel

baseurl=http://mirrors.aliyun.com/epel/7/x86_64/

gpgcheck=0

[root@TomcatA ~]#

[root@TomcatA ~]# yum install -y tomcat

[root@TomcatA ~]# mkdir -pv/usr/share/tomcat/webapps/ROOT/WEB-INF

mkdir: created directory‘/usr/share/tomcat/webapps/ROOT’

mkdir: created directory‘/usr/share/tomcat/webapps/ROOT/WEB-INF’

[root@TomcatA ~]# vim/usr/share/tomcat/webapps/ROOT/index.jsp

[root@TomcatA ~]# cat/usr/share/tomcat/webapps/ROOT/index.jsp

<%@ page language="java" %>
<html>
         <head><title>TomcatA</title></head>
         <body>
                   <h1><fontcolor="red">TomcatA</font></h1>
                   <table                border="1">
                            <tr>
                                     <td>SessionID</td>
                            <%session.setAttribute("test.com","test.com"); %>
                                     <td><%=session.getId() %></td>
                            </tr>
                            <tr>
                                     <td>Createdon</td>
                                     <td><%=session.getCreationTime() %></td>
                            </tr>
                   </table>
         </body>
</html>

[root@TomcatA ~]# scp/usr/share/tomcat/webapps/ROOT/index.jsp 192.168.0.21:/usr/share/tomcat/webapps/ROOT/

The authenticity of host '192.168.0.21(192.168.0.21)' can't be established.

ECDSA key fingerprint isfc:7f:15:2b:2d:69:c9:c1:b0:79:7c:ce:ab:c0:0c:36.

Are you sure you want to continueconnecting (yes/no)? yes

Warning: Permanently added '192.168.0.21'(ECDSA) to the list of known hosts.

root@192.168.0.21's password:

index.jsp 100% 410 0.4KB/s 00:00

[root@TomcatA ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 128 :::22 :::*

[root@TomcatA ~]# systemctl start tomcat

[root@TomcatA ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

[root@TomcatA ~]#

TomcatB环境配置

[root@TomcatB ~]# ifconfig

eno16777736:flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500

inet 192.168.0.21 netmask255.255.255.0 broadcast 192.168.0.255

inet6 fe80::20c:29ff:fed8:be3b prefixlen 64 scopeid0x20<link>

ether 00:0c:29:d8:be:3b txqueuelen 1000 (Ethernet)

RX packets 839 bytes 107014(104.5 KiB)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 430 bytes 68356 (66.7KiB)

TX errors 0 dropped 0 overruns0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536

inet 127.0.0.1 netmask 255.0.0.0

inet6 ::1 prefixlen 128 scopeid 0x10<host>

loop txqueuelen 0 (Local Loopback)

RX packets 0 bytes 0 (0.0 B)

RX errors 0 dropped 0 overruns 0 frame 0

TX packets 0 bytes 0 (0.0 B)

TX errors 0 dropped 0 overruns0 carrier 0 collisions 0

[root@TomcatB ~]# cat/etc/yum.repos.d/hehe.repo

[cdrom-repo]

name=cdrom-repo

baseurl=file:///cdrom

gpgcheck=0

[aliyun-epel]

name=aliyun-epel

baseurl=http://mirrors.aliyun.com/epel/7/x86_64/

gpgcheck=0

[root@TomcatB ~]#

[root@TomcatB ~]# yum install -y tomcat

[root@TomcatB ~]# mkdir -pv/usr/share/tomcat/webapps/ROOT/WEB-INF

mkdir: created directory‘/usr/share/tomcat/webapps/ROOT’

mkdir: created directory‘/usr/share/tomcat/webapps/ROOT/WEB-INF’

[root@TomcatB ~]# vim/usr/share/tomcat/webapps/ROOT/index.jsp

[root@TomcatB ~]# cat/usr/share/tomcat/webapps/ROOT/index.jsp

<%@ page language="java" %>
<html>
         <head><title>TomcatB</title></head>
         <body>
                   <h1><fontcolor="blue">TomcatB</font></h1>
                   <table                border="1">
                            <tr>
                                     <td>SessionID</td>
                            <%session.setAttribute("test.com","test.com"); %>
                                     <td><%=session.getId() %></td>
                            </tr>
                            <tr>
                                     <td>Createdon</td>
                                     <td><%=session.getCreationTime() %></td>
                            </tr>
                   </table>
         </body>
</html>

[root@TomcatB ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 128 :::22 :::*

[root@TomcatB ~]# systemctl start tomcat

[root@TomcatB ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

[root@TomcatB ~]#

测试两台tomcat服务器工作是否正常

使用浏览器访问结果如图所示

接下来就需要配置前端的反代服务器，以及tomcat集群session复制了

httpd反代tomcat集群配置

在这里我们为了节约资源就把httpd反代服务器配置到TomcatA服务器上

[root@TomcatA ~]# yum install -y httpd

[root@TomcatA ~]# vim/etc/httpd/conf.d/httpd_tomcat.conf

[root@TomcatA ~]# cat/etc/httpd/conf.d/httpd_tomcat.conf

#LoadModule proxy_modulemodules/mod_proxy.so
#LoadModule proxy_ajp_modulemodules/mod_proxy_ajp.so
#LoadModule proxy_balancer_modulemodules/mod_proxy_balancer.so
 
<proxy balancer://tomcatcluster>
         BalancerMemberajp://192.168.0.20:8009 route=TomcatA loadfactor=1
         BalancerMemberajp://192.168.0.21:8009 route=TomcatB loadfactor=1
         ProxySetlbmethod=byrequests
         #ProxySetstickysession=JSESSIONID
</Proxy>
 
<VirtualHost *:80>
         ServerNameproxy.test.com
         ProxyViaOn
         ProxyRequestsOff
         ProxyPreserveHostOn
         <Proxy*>
                   Requireall granted
         </Proxy>
         ProxyPass/ balancer://tomcatcluster/
         ProxyPa***everse/ balancer://tomcatcluster/
         <Location/>
                   Requireall granted
         </Location>
         <Location/balancer-manager>
                   SetHandlerbalancer-manager
                   ProxyPass!
                   Requireall granted
         </Location>
</VirtualHost>

[root@TomcatA ~]# vim/etc/httpd/conf/httpd.conf

修改如下内容

ServerName www.example.com:80
<IfModule dir_module>
   DirectoryIndex index.jsp
</IfModule>

[root@TomcatA ~]# httpd -t

Syntax OK

[root@TomcatA ~]# cp/etc/tomcat/server.xml{,.bak}

[root@TomcatA ~]# vim/etc/tomcat/server.xml

默认配置文件需要修改以下几个地方

Engine标签下添加集群配置内容

   <ClusterclassName="org.apache.catalina.ha.tcp.SimpleTcpCluster"
           channelSendOptions="8">
 
       <ManagerclassName="org.apache.catalina.ha.session.DeltaManager"
           expireSessionsOnShutdown="false"
            notifyListenersOnReplication="true"/>
 
       <ChannelclassName="org.apache.catalina.tribes.group.GroupChannel">
           <MembershipclassName="org.apache.catalina.tribes.membership.McastService"
                address="228.0.0.4"
               port="45564"
                frequency="500"
                dropTime="3000"/>
           <ReceiverclassName="org.apache.catalina.tribes.transport.nio.NioReceiver"
                address="192.168.0.20"
                port="4000"
                autoBind="100"
               selectorTimeout="5000"
                maxThreads="6"/>
 
           <SenderclassName="org.apache.catalina.tribes.transport.ReplicationTransmitter">
                <TransportclassName="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
           </Sender>
           <InterceptorclassName="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
           <InterceptorclassName="org.apache.catalina.tribes.group.interceptors.MessageDispatch15Interceptor"/>
       </Channel>
 
       <ValveclassName="org.apache.catalina.ha.tcp.ReplicationValve"
           filter=""/>
       <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>
 
       <DeployerclassName="org.apache.catalina.ha.deploy.FarmWarDeployer"
           tempDir="/tmp/war-temp/"
           deployDir="/tmp/war-deploy/"
           watchDir="/tmp/war-listen/"
           watchEnabled="false"/>
 
       <ClusterListenerclassName="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener"/>
       <ClusterListenerclassName="org.apache.catalina.ha.session.ClusterSessionListener"/>
</Cluster>

修改日志格式为httpd combined格式

<Valve className="org.apache.catalina.valves.AccessLogValve"directory="logs"
      prefix="localhost_access_log."suffix=".txt"
      pattern="%h %l %u %t&quot;%r&quot; %s %b &quot;%{Referer}i&quot;
            &quot;%{User-Agent}i&quot;" />

[root@TomcatB ~]# mv/etc/tomcat/server.xml{,.bak}

[root@TomcatA ~]# scp/etc/tomcat/server.xml 192.168.0.21:/etc/tomcat/

root@192.168.0.21's password:

server.xml 100% 8226 8.0KB/s 00:00

[root@TomcatA ~]#

对比TomcatA需要修改的地方

<Engine name="Catalina"defaultHost="localhost" jvmRoute="TomcatB">
           <ReceiverclassName="org.apache.catalina.tribes.transport.nio.NioReceiver"
               address="192.168.0.21"

添加应用程序自己的web.xml配置文件，需要有<distributable/>这个标签

[root@TomcatA ~]# vim/usr/share/tomcat/webapps/ROOT/WEB-INF/web.xml

[root@TomcatA ~]# cat/usr/share/tomcat/webapps/ROOT/WEB-INF/web.xml

<?xml version="1.0"encoding="ISO-8859-1"?>
 
<web-appxmlns="http://java.sun.com/xml/ns/javaee"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
                     http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
 version="3.0"
 metadata-complete="true">
 
         <distributable/>
 
</web-app>

[root@TomcatA ~]# scp/usr/share/tomcat/webapps/ROOT/WEB-INF/web.xml192.168.0.21:/usr/share/tomcat/webapps/ROOT/WEB-INF

root@192.168.0.21's password:

web.xml 100% 356 0.4KB/s 00:00

[root@TomcatA ~]# systemctl restart tomcat

[root@TomcatA ~]# systemctl start httpd

[root@TomcatA ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port Peer Address:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 128 :::80 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

LISTEN 0 50 ::ffff:192.168.0.20:4000 :::*

LISTEN 0 1 ::ffff:127.0.0.1:8005 :::*

[root@TomcatA ~]#

TomcatA重启成功日志如下

[root@TomcatA ~]# tail -f/var/log/tomcat/catalina.2017-09-26.log

Sep 26, 2017 2:57:34 AMorg.apache.catalina.core.StandardServer await

INFO: A valid shutdown command was receivedvia the shutdown port. Stopping the Server instance.

Sep 26, 2017 2:57:34 AMorg.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler["http-bio-8080"]

Sep 26, 2017 2:57:35 AMorg.apache.catalina.core.AprLifecycleListener init

INFO: The APR based Apache Tomcat Nativelibrary which allows optimal performance in production environments was notfound on the java.library.path:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib

Sep 26, 2017 2:57:35 AMorg.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler["http-bio-8080"]

Sep 26, 2017 2:57:35 AMorg.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler["ajp-bio-8009"]

Sep 26, 2017 2:57:35 AMorg.apache.catalina.startup.Catalina load

INFO: Initialization processed in 379 ms

Sep 26, 2017 2:57:35 AMorg.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Sep 26, 2017 2:57:35 AMorg.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: ApacheTomcat/7.0.54

Sep 26, 2017 2:57:35 AM org.apache.catalina.ha.tcp.SimpleTcpClusterstartInternal

INFO: Cluster is about to start

Sep 26, 2017 2:57:35 AMorg.apache.catalina.tribes.transport.ReceiverBase bind

INFO: Receiver Server Socket boundto:/192.168.0.20:4000

Sep 26, 2017 2:57:35 AM org.apache.catalina.tribes.membership.McastServiceImplsetupSocket

INFO: Setting cluster mcast soTimeout to500

Sep 26, 2017 2:57:35 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Sleeping for 1000 milliseconds toestablish cluster membership, start level:4

Sep 26, 2017 2:57:36 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Done sleeping, membershipestablished, start level:4

Sep 26, 2017 2:57:36 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Sleeping for 1000 milliseconds toestablish cluster membership, start level:8

Sep 26, 2017 2:57:37 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Done sleeping, membershipestablished, start level:8

Sep 26, 2017 2:57:37 AMorg.apache.catalina.ha.deploy.FarmWarDeployer start

SEVERE: FarmWarDeployer can only work ashost cluster subelement!

Sep 26, 2017 2:57:37 AMorg.apache.catalina.startup.HostConfig deployDirectory

INFO: Deploying web application directory/var/lib/tomcat/webapps/ROOT

Sep 26, 2017 2:58:13 AMorg.apache.catalina.util.SessionIdGenerator createSecureRandom

INFO: Creation of SecureRandom instance forsession ID generation using [SHA1PRNG] took [35,019] milliseconds.

Sep 26, 2017 2:58:13 AMorg.apache.catalina.ha.session.DeltaManager startInternal

INFO: Register manager localhost# tocluster element Engine with name Catalina

Sep 26, 2017 2:58:13 AMorg.apache.catalina.ha.session.DeltaManager startInternal

INFO: Starting clustering manager atlocalhost#

Sep 26, 2017 2:58:13 AMorg.apache.catalina.ha.session.DeltaManager getAllClusterSessions

INFO: Manager [localhost#]: skipping statetransfer. No members active in cluster group.

Sep 26, 2017 2:58:13 AMorg.apache.catalina.startup.HostConfig deployDirectory

INFO: Deployment of web applicationdirectory /var/lib/tomcat/webapps/ROOT has finished in 35,563 ms

Sep 26, 2017 2:58:13 AMorg.apache.catalina.ha.session.JvmRouteBinderValve startInternal

INFO: JvmRouteBinderValve started

Sep 26, 2017 2:58:13 AMorg.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler["http-bio-8080"]

Sep 26, 2017 2:58:13 AMorg.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler["ajp-bio-8009"]

Sep 26, 2017 2:58:13 AM org.apache.catalina.startup.Catalinastart

INFO: Server startup in 37684 ms

TomcatB重启成功后TomcatA会产生如下日志

Sep 26, 2017 3:01:05 AMorg.apache.catalina.tribes.io.BufferPool getBufferPool

INFO: Created a buffer pool with maxsize:104857600 bytes of type:org.apache.catalina.tribes.io.BufferPool15Impl

Sep 26, 2017 3:01:06 AMorg.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded

INFO: Replication memberadded:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 0,21}:4000,{192, 168, 0, 21},4000, alive=1014, securePort=-1, UDP Port=-1, id={5025 -81 7 -94 116 76 51 -100 -57 116 -83 10 93 -110 -108 }, payload={},command={}, domain={}, ]

TomcatB需做如下修改

[root@TomcatB ~]# cat/usr/share/tomcat/webapps/ROOT/WEB-INF/web.xml

<?xml version="1.0"encoding="ISO-8859-1"?>
 
<web-appxmlns="http://java.sun.com/xml/ns/javaee"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
                     http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
 version="3.0"
  metadata-complete="true">
 
         <distributable/>
 
</web-app>

[root@TomcatB ~]# systemctl restart tomcat

TomcatB成功重启过程日志如下

[root@TomcatB ~]# tail -f/var/log/tomcat/catalina.2017-09-26.log

Sep 26, 2017 3:01:03 AMorg.apache.catalina.core.StandardServer await

INFO: A valid shutdown command was receivedvia the shutdown port. Stopping the Server instance.

Sep 26, 2017 3:01:03 AMorg.apache.coyote.AbstractProtocol pause

INFO: Pausing ProtocolHandler["http-bio-8080"]

Sep 26, 2017 3:01:04 AMorg.apache.catalina.core.AprLifecycleListener init

Sep 26, 2017 3:01:04 AMorg.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler["http-bio-8080"]

Sep 26, 2017 3:01:04 AMorg.apache.coyote.AbstractProtocol init

INFO: Initializing ProtocolHandler["ajp-bio-8009"]

Sep 26, 2017 3:01:04 AM org.apache.catalina.startup.Catalinaload

INFO: Initialization processed in 413 ms

Sep 26, 2017 3:01:04 AMorg.apache.catalina.core.StandardService startInternal

INFO: Starting service Catalina

Sep 26, 2017 3:01:04 AMorg.apache.catalina.core.StandardEngine startInternal

INFO: Starting Servlet Engine: ApacheTomcat/7.0.54

Sep 26, 2017 3:01:04 AMorg.apache.catalina.ha.tcp.SimpleTcpCluster startInternal

INFO: Cluster is about to start

Sep 26, 2017 3:01:04 AMorg.apache.catalina.tribes.transport.ReceiverBase bind

INFO: Receiver Server Socket boundto:/192.168.0.21:4000

Sep 26, 2017 3:01:04 AMorg.apache.catalina.tribes.membership.McastServiceImpl setupSocket

INFO: Setting cluster mcast soTimeout to500

Sep 26, 2017 3:01:04 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Sleeping for 1000 milliseconds toestablish cluster membership, start level:4

Sep 26, 2017 3:01:05 AMorg.apache.catalina.ha.tcp.SimpleTcpCluster memberAdded

INFO: Replication memberadded:org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 0,20}:4000,{192, 168, 0, 20},4000, alive=209565, securePort=-1, UDP Port=-1,id={6 -86 -55 -67 102 127 77 -57 -67 -67 -105 -20 -37 -55 76 -22 }, payload={},command={}, domain={}, ]

Sep 26, 2017 3:01:05 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Done sleeping, membershipestablished, start level:4

Sep 26, 2017 3:01:05 AMorg.apache.catalina.tribes.membership.McastServiceImpl waitForMembers

INFO: Sleeping for 1000 milliseconds toestablish cluster membership, start level:8

Sep 26, 2017 3:01:05 AMorg.apache.catalina.tribes.io.BufferPool getBufferPool

INFO: Created a buffer pool with maxsize:104857600 bytes of type:org.apache.catalina.tribes.io.BufferPool15Impl

Sep 26, 2017 3:01:06 AM org.apache.catalina.tribes.membership.McastServiceImplwaitForMembers

INFO: Done sleeping, membershipestablished, start level:8

Sep 26, 2017 3:01:06 AMorg.apache.catalina.ha.deploy.FarmWarDeployer start

SEVERE: FarmWarDeployer can only work ashost cluster subelement!

Sep 26, 2017 3:01:06 AMorg.apache.catalina.startup.HostConfig deployDirectory

INFO: Deploying web application directory/var/lib/tomcat/webapps/ROOT

Sep 26, 2017 3:01:48 AMorg.apache.catalina.util.SessionIdGenerator createSecureRandom

INFO: Creation of SecureRandom instance forsession ID generation using [SHA1PRNG] took [41,566] milliseconds.

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.DeltaManager startInternal

INFO: Register manager localhost# tocluster element Engine with name Catalina

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.DeltaManager startInternal

INFO: Starting clustering manager atlocalhost#

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.DeltaManager getAllClusterSessions

INFO: Manager [localhost#], requestingsession state from org.apache.catalina.tribes.membership.MemberImpl[tcp://{192,168, 0, 20}:4000,{192, 168, 0, 20},4000, alive=253179, securePort=-1, UDPPort=-1, id={6 -86 -55 -67 102 127 77 -57 -67 -67 -105 -20 -37 -55 76 -22 },payload={}, command={}, domain={}, ]. This operation will timeout if no sessionstate has been received within 60 seconds.

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.DeltaManager waitForSendAllSessions

INFO: Manager [localhost#]; session statesend at 9/26/17 3:01 AM received in 114 ms.

Sep 26, 2017 3:01:48 AMorg.apache.catalina.startup.HostConfig deployDirectory

INFO: Deployment of web applicationdirectory /var/lib/tomcat/webapps/ROOT has finished in 42,286 ms

Sep 26, 2017 3:01:48 AMorg.apache.catalina.ha.session.JvmRouteBinderValve startInternal

INFO: JvmRouteBinderValve started

Sep 26, 2017 3:01:48 AMorg.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler["http-bio-8080"]

Sep 26, 2017 3:01:48 AMorg.apache.coyote.AbstractProtocol start

INFO: Starting ProtocolHandler["ajp-bio-8009"]

Sep 26, 2017 3:01:48 AMorg.apache.catalina.startup.Catalina start

INFO: Server startup in 44401 ms

集群session配置成功，使用浏览器访问反代服务器的80端口检验效果

至此，httpd反代tomcat集群，并且tomcat集群session复制都成功实现了

Nginx反代tomat集群配置

在这里我们为了节约资源就把nginx反代服务器配置到TomcatB服务器上

[root@TomcatB ~]# yum install -y nginx

[root@TomcatB ~]# cp/etc/nginx/nginx.conf{,.bak}

[root@TomcatB ~]# vim /etc/nginx/nginx.conf

Nginx默认配置文件需要修改以下几处

定义后端tomcat集群

upstreamtomcatservers {
       server 192.168.0.20:8080;
       server 192.168.0.21:8080;
}

注释IPV6监听

   server {
       listen       80 default_server;
       #listen       [::]:80default_server;

定义完全反代至后端tomcat集群

       location / {
           root  /usr/share/tomcat/wepapps/ROOT;
            index  index.jsp;
           proxy_pass http://tomcatservers;
       }

定义让后端tomcat集群获取客户端真实IP

       proxy_set_header Host $host;
       proxy_set_header X-Real-IP $remote_addr;
       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

[root@TomcatB ~]# nginx -t

nginx: the configuration file/etc/nginx/nginx.conf syntax is ok

nginx: configuration file /etc/nginx/nginx.conftest is successful

[root@TomcatB ~]#

继httpd反代tomcat集群配置之后需要修改的地方

[root@TomcatA ~]# vim/etc/tomcat/server.xml

       <ValveclassName="org.apache.catalina.valves.AccessLogValve"directory="logs"
              prefix="localhost_access_log." suffix=".txt"
              pattern="%{X-Forwarded-For}i %l %u %t &quot;%r&quot; %s %b&quot;%{Referer}i&quot;
                      &quot;%{User-Agent}i&quot;"/>
               <!--pattern="%h %l %u %t&quot;%r&quot; %s %b &quot;%{Referer}i&quot;
                       &quot;%{User-Agent}i&quot;"/> -->

[root@TomcatA ~]# systemctl restart tomcat

[root@TomcatB ~]# vim/etc/tomcat/server.xml

       <ValveclassName="org.apache.catalina.valves.AccessLogValve"directory="logs"
              prefix="localhost_access_log." suffix=".txt"
              pattern="%{X-Forwarded-For}i %l %u %t &quot;%r&quot; %s %b&quot;%{Referer}i&quot;
                      &quot;%{User-Agent}i&quot;"/>
               <!--pattern="%h %l %u %t&quot;%r&quot; %s %b &quot;%{Referer}i&quot;
                       &quot;%{User-Agent}i&quot;"/> -->

[root@TomcatB ~]# systemctl restart tomcat

Httpd 反代tomcat集群无需在httpd添加额外传递客户端ip的参数以及修改tomcat日志记录格式就可以让后端tomcat记录客户端的真实IP，但nginx反代tomcat集群时必须首先在nginx上添加向后端tomat服务器传递客户端真实IP的参数，其次还需要修改tomcat服务器的默认日志记录格式，才能成功记录客户端的真实IP

[root@TomcatB ~]# ss -tnl

State Recv-Q Send-Q LocalAddress:Port PeerAddress:Port

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

LISTEN 0 50 ::ffff:192.168.0.21:4000 :::*

LISTEN 0 1 ::ffff:127.0.0.1:8005 :::*

[root@TomcatB ~]# systemctl start nginx

[root@TomcatB ~]# ss -tnl

State Recv-Q Send-Q Local Address:Port Peer Address:Port

LISTEN 0 128 *:80 *:*

LISTEN 0 128 *:22 *:*

LISTEN 0 100 :::8009 :::*

LISTEN 0 100 :::8080 :::*

LISTEN 0 128 :::22 :::*

LISTEN 0 50 ::ffff:192.168.0.21:4000 :::*

LISTEN 0 1 ::ffff:127.0.0.1:8005 :::*

[root@TomcatB ~]#

Nginx反代tomcat集群session复制效果检验

这里主要是为了给大家展示tomcat集群session复制的效果，所以在httpd或者nginx反代tomat集群这方面并没有给大家展示更多情况，httpd反代tomcat集群可以使用http协议、ajp协议、mod_jk协议三种方式反代，httpd反代tomcat集群时可以在实现session复制的同时，实现session粘性，nginx反代tomcat集群时还可以配置权重、健康状态检查、sorry server，在这里不做全面展示，有需求者可查阅本博客相应专业板块。

来源：oschina

链接：https://my.oschina.net/u/4351661/blog/4341728

标签

apache

apache tribes

Java EE