How to configure Hazelcast for session caching using spring while limiting it to a set of nodes?
I need to set up Hazelcast session caching using Spring. Using the configuration from the hazelcast docs is simple. However, its insufficient. I need to use a different c
-
Thank you for the suggested answer. However, I think I've solved this using the following configuration. I would appreciate any feedback from anyone about this configuration.
My approach:
1) Establish the instance with a spring configuration. 2) Enhance the instance with a minimally configured hazelcast.xml file with the Web filter configuration. Note that mulitcast and tcp-ip joiners are false.
web.xml:
<?xml version="1.0" encoding="UTF-8"?> <web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xmlns:web="http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" id="WebApp_ID" version="3.0"> <!-- The definition of the Root Spring Container shared by all Servlets and Filters --> <context-param> <param-name>contextConfigLocation</param-name> <param-value> /WEB-INF/spring/servlet-context.xml, /WEB-INF/spring/root-context.xml, .... /WEB-INF/spring/hazelcastContext.xml </param-value> </context-param> <listener> <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class> </listener> .... <filter> <filter-name>hazelcast-filter</filter-name> <filter-class>com.hazelcast.web.WebFilter</filter-class> <!-- Name of the distributed map storing your web session objects --> <init-param> <param-name>map-name</param-name> <param-value>my-sessions</param-value> </init-param> <!-- How is your load-balancer configured? stick-session means all requests of a session is routed to the node where the session is first created. This is excellent for performance. If sticky-session is set to false, when a session is updated on a node, entry for this session on all other nodes is invalidated. You have to know how your load-balancer is configured before setting this parameter. Default is true. --> <init-param> <param-name>sticky-session</param-name> <param-value>true</param-value> </init-param> <!-- Name of session id cookie --> <init-param> <param-name>cookie-name</param-name> <param-value>hazelcast.sessionId</param-value> </init-param> <!-- Domain of session id cookie. Default is based on incoming request. --> <init-param> <param-name>cookie-domain</param-name> <param-value>.mycompany.com</param-value> </init-param> <!-- Should cookie only be sent using a secure protocol? Default is false. --> <init-param> <param-name>cookie-secure</param-name> <param-value>false</param-value> </init-param> <!-- Should HttpOnly attribute be set on cookie ? Default is false. --> <init-param> <param-name>cookie-http-only</param-name> <param-value>false</param-value> </init-param> <!-- Are you debugging? Default is false. --> <init-param> <param-name>debug</param-name> <param-value>true</param-value> </init-param> <!-- Configuration xml location; * as servlet resource OR * as classpath resource OR * as URL Default is one of hazelcast-default.xml or hazelcast.xml in classpath. --> <init-param> <param-name>config-location</param-name> <param-value>/WEB-INF/classes/hazelcast.xml</param-value> </init-param> <!-- Do you want to use an existing HazelcastInstance? Default is null. -->`enter code here` <init-param> <param-name>instance-name</param-name> <param-value>myapp</param-value> </init-param> <!-- Do you want to connect as a client to an existing cluster? Default is false. --> <init-param> <param-name>use-client</param-name> <param-value>false</param-value> </init-param> <!-- Client configuration location; * as servlet resource OR * as classpath resource OR * as URL Default is null. --> <init-param> <param-name>client-config-location</param-name> <param-value>/WEB-INF/classes/hazelcast-client.properties</param-value> </init-param> <!-- Do you want to shutdown HazelcastInstance during web application undeploy process? Default is true. --> <init-param> <param-name>shutdown-on-destroy</param-name> <param-value>true</param-value> </init-param> </filter> <filter-mapping> <filter-name>hazelcast-filter</filter-name> <url-pattern>/*</url-pattern> <dispatcher>FORWARD</dispatcher> <dispatcher>INCLUDE</dispatcher> <dispatcher>REQUEST</dispatcher> </filter-mapping> ... </web-app>hazelcast.xml (copied from hazelcast.xml inside jar file)...
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-config-2.4.xsd" xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> ... <network> <port auto-increment="true">5701</port> <outbound-ports> <!-- Allowed port range when connecting to other nodes. 0 or * means use system provided port. --> <ports>0</ports> </outbound-ports> <join> <multicast enabled="false"> <multicast-group>224.2.2.3</multicast-group> <multicast-port>54327</multicast-port> </multicast> <tcp-ip enabled="false"> <interface>127.0.0.1</interface> </tcp-ip> ... </join> ... </network> ... </hazelcast>Spring config....
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:hz="http://www.hazelcast.com/schema/spring" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.hazelcast.com/schema/spring http://www.hazelcast.com/schema/spring/hazelcast-spring-2.4.xsd"> <bean id="hazelcast" class="com.hazelcast.core.Hazelcast" /> <!-- Hazelcast Instance configuration --> <hz:hazelcast id="myapp"> <hz:config> <!-- Hazelcast Instance Name --> <hz:instance-name>${hz.instance.name}</hz:instance-name> <!-- Hazelcast Group Name and Password --> <hz:group name="${hz.group.name}" password="${hz.group.password}" /> <!-- Hazelcast Management Center URL --> <hz:management-center enabled="${hz.management.center.enabled}" url="${hz.management.center.url}" /> <!-- Hazelcast Tcp based network configuration --> <hz:network port="${hz.network.port}" port-auto-increment="${hz.network.port.auto.increment}"> <hz:join> <hz:multicast enabled="${hz.multicast.enabled}" multicast-group="224.2.2.3" multicast-port="54327" /> <hz:tcp-ip enabled="${hz.tcp.ip.enabled}"> <hz:members>${hz.members}</hz:members> </hz:tcp-ip> </hz:join> </hz:network> <!-- Hazelcast Distributed Map configuration --> <hz:map name="map" backup-count="${hz.map.backup.count}" max-size="${hz.map.max.size}" eviction-percentage="${hz.map.eviction.percentage}" read-backup-data="${hz.map.read.backup.data}" eviction-policy="${hz.map.eviction.policy}" merge-policy="${hz.map.merge.policy}" /> </hz:config> </hz:hazelcast>Property file....
#-- Hazelcast properties. hz.instance.name = myapp hz.group.name = CERT hz.group.password = cert hz.management.center.enabled = true hz.management.center.url = http://127.0.0.1:8080/mancenter hz.network.port = 5701 hz.network.port.auto.increment = true hz.multicast.enabled = true hz.tcp.ip.enabled = false hz.members = 127.0.0.1 hz.executor.service.core.pool.size = 2 hz.executor.service.max.pool.size = 30 hz.executor.service.keep.alive.seconds = 30 hz.map.backup.count=2 hz.map.max.size=0 hz.map.eviction.percentage=30 hz.map.read.backup.data=true hz.map.cache.value=true hz.map.eviction.policy=NONE hz.map.merge.policy=hz.ADD_NEW_ENTRYRoot Context.xml
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:hz="http://www.hazelcast.com/schema/spring" xmlns:jee="http://www.springframework.org/schema/jee" xmlns:context="http://www.springframework.org/schema/context" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.1.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.1.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-3.1.xsd http://www.hazelcast.com/schema/spring http://www.hazelcast.com/schema/spring/hazelcast-spring-2.4.xsd"> ... <bean id="propertyConfigurer" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer"> <property name="locations"> <list> <jee:jndi-lookup jndi-name="java:comp/env/config_file" /> </list> </property> </bean> ... </beans>Tomcat Config...
<Host appBase="webapps" autoDeploy="true" name="localhost" unpackWARs="true"> ... <Context docBase="myapp" path="/myapp" reloadable="true" source="org.eclipse.jst.j2ee.server:pwc-ws"> <Environment description="" name="config_file" override="false" type="java.lang.String" value="file:c:/path/to/config/myapp.properties" /> </Context> ... </Host>Tomcat output (Note: This is a restart scenario of a two node hazelcast group. In this scenario, Node 1 is restarted. The output of Node 2 shows the drop of node 1 from the group and then its return to the group).
Node 1 of Group CERT
Nov 19, 2013 4:27:56 PM com.hazelcast.impl.AddressPicker INFO: Prefer IPv4 stack is true. Nov 19, 2013 4:27:56 PM com.hazelcast.impl.AddressPicker INFO: Picked Address[10.23.43.13]:5701, using socket ServerSocket[addr=/0:0:0:0:0:0:0:0,localport=5701], bind any local is true Nov 19, 2013 4:27:57 PM com.hazelcast.system INFO: [10.23.43.13]:5701 [CERT] Hazelcast Community Edition 2.4 (20121017) starting at Address[10.23.43.13]:5701 Nov 19, 2013 4:27:57 PM com.hazelcast.system INFO: [10.23.43.13]:5701 [CERT] Copyright (C) 2008-2012 Hazelcast.com Nov 19, 2013 4:27:57 PM com.hazelcast.impl.LifecycleServiceImpl INFO: [10.23.43.13]:5701 [CERT] Address[10.23.43.13]:5701 is STARTING Nov 19, 2013 4:27:57 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:27:57 PM com.hazelcast.impl.MulticastJoiner INFO: [10.23.43.13]:5701 [CERT] Connecting to master node: Address[10.23.43.14]:5701 Nov 19, 2013 4:27:57 PM com.hazelcast.nio.ConnectionManager INFO: [10.23.43.13]:5701 [CERT] 54106 accepted socket connection from /10.23.43.14:5701 Nov 19, 2013 4:27:57 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:27:57 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:27:58 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:27:58 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:27:59 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:27:59 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:28:00 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:28:00 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:28:01 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:28:01 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:28:02 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:28:02 PM com.hazelcast.impl.Node INFO: [10.23.43.13]:5701 [CERT] ** setting master address to Address[10.23.43.14]:5701 Nov 19, 2013 4:28:02 PM com.hazelcast.cluster.ClusterManager INFO: [10.23.43.13]:5701 [CERT] Members [2] { Member [10.23.43.14]:5701 Member [10.23.43.13]:5701 this } Nov 19, 2013 4:28:04 PM com.hazelcast.impl.LifecycleServiceImpl INFO: [10.23.43.13]:5701 [CERT] Address[10.23.43.13]:5701 is STARTED Nov 19, 2013 4:28:04 PM com.hazelcast.impl.management.ManagementCenterService INFO: [10.23.43.13]:5701 [CERT] Hazelcast will connect to Management Center on address: http://localhost:8080/mancenter/ Nov 19, 2013 4:28:04 PM com.hazelcast.config.UrlXmlConfig INFO: Configuring Hazelcast from 'jndi:/localhost/pwc-ui/WEB-INF/classes/hazelcast.xml'. Nov 19, 2013 4:28:04 PM com.hazelcast.web.WebFilter INFO: sticky:true, debug: true, shutdown-on-destroy: true, map-name: my-sessions Nov 19, 2013 4:28:05 PM org.apache.catalina.startup.HostConfig deployDescriptorNode 2 of Group CERT (note the drop and re-add)
Nov 19, 2013 4:27:11 PM com.hazelcast.nio.Connection INFO: [10.23.43.14]:5701 [CERT] Connection [Address[10.23.43.13]:5701] lost. Reason: java.io.IOException[Connection reset by peer] Nov 19, 2013 4:27:11 PM com.hazelcast.nio.ReadHandler WARNING: [10.23.43.14]:5701 [CERT] hz.pwc.IO.thread-1 Closing socket to endpoint Address[10.23.43.13]:5701, Cause:java.io.IOException: Connection reset by peer Nov 19, 2013 4:27:12 PM com.hazelcast.nio.SocketConnector INFO: [10.23.43.14]:5701 [CERT] Could not connect to: /10.23.43.13:5701. Reason: ConnectException[Connection refused] Nov 19, 2013 4:27:13 PM com.hazelcast.nio.SocketConnector INFO: [10.23.43.14]:5701 [CERT] Could not connect to: /10.23.43.13:5701. Reason: ConnectException[Connection refused] Nov 19, 2013 4:27:14 PM com.hazelcast.nio.SocketConnector INFO: [10.23.43.14]:5701 [CERT] Could not connect to: /10.23.43.13:5701. Reason: ConnectException[Connection refused] Nov 19, 2013 4:27:14 PM com.hazelcast.nio.ConnectionMonitor WARNING: [10.23.43.14]:5701 [CERT] Removing connection to endpoint Address[10.23.43.13]:5701 Cause => java.net.ConnectException {Connection refused}, Error-Count: 5 Nov 19, 2013 4:27:14 PM com.hazelcast.cluster.ClusterManager INFO: [10.23.43.14]:5701 [CERT] Removing Address Address[10.23.43.13]:5701 Nov 19, 2013 4:27:14 PM com.hazelcast.impl.PartitionManager INFO: [10.23.43.14]:5701 [CERT] Starting to send partition replica diffs...true Nov 19, 2013 4:27:14 PM com.hazelcast.cluster.ClusterManager INFO: [10.23.43.14]:5701 [CERT] Members [1] { Member [10.23.43.14]:5701 this } Nov 19, 2013 4:27:18 PM com.hazelcast.impl.PartitionManager INFO: [10.23.43.14]:5701 [CERT] Total 0 partition replica diffs have been processed. Nov 19, 2013 4:27:18 PM com.hazelcast.impl.PartitionManager INFO: [10.23.43.14]:5701 [CERT] Re-partitioning cluster data... Immediate-Tasks: 0, Scheduled-Tasks: 0 Nov 19, 2013 4:27:57 PM com.hazelcast.nio.SocketAcceptor INFO: [10.23.43.14]:5701 [CERT] 5701 is accepting socket connection from /10.23.43.13:54106 Nov 19, 2013 4:27:57 PM com.hazelcast.nio.ConnectionManager INFO: [10.23.43.14]:5701 [CERT] 5701 accepted socket connection from /10.23.43.13:54106 Nov 19, 2013 4:27:57 PM com.hazelcast.web.WebFilter INFO: Created new session with id: HZ650FDF62693F45A99AC0C30BBD8840B0 Nov 19, 2013 4:27:57 PM com.hazelcast.web.WebFilter INFO: 195 is sessions.size and originalSessions.size: 195 Nov 19, 2013 4:27:57 PM com.hazelcast.web.WebFilter INFO: PUTTING SESSION HZ650FDF62693F45A99AC0C30BBD8840B0 Nov 19, 2013 4:28:02 PM com.hazelcast.cluster.ClusterManager INFO: [10.23.43.14]:5701 [CERT] Members [2] { Member [10.23.43.14]:5701 this Member [10.23.43.13]:5701 } Nov 19, 2013 4:28:02 PM com.hazelcast.impl.PartitionManager INFO: [10.23.43.14]:5701 [CERT] Re-partitioning cluster data... Immediate-Tasks: 271, Scheduled-Tasks: 0 Nov 19, 2013 4:28:24 PM com.hazelcast.web.WebFilter INFO: Created new session with id: HZAD50E5F483CC448C9FA7CB66D65848BB Nov 19, 2013 4:28:24 PM com.hazelcast.web.WebFilter INFO: 196 is sessions.size and originalSessions.size: 196 Nov 19, 2013 4:28:24 PM com.hazelcast.web.WebFilter INFO: PUTTING SESSION HZAD50E5F483CC448C9FA7CB66D65848BB Nov 19, 2013 4:28:24 PM com.hazelcast.web.WebFilter INFO: Request is instance of RequestWrapper! Continue... Nov 19, 2013 4:28:24 PM com.hazelcast.web.WebFilter INFO: Request is instance of RequestWrapper! Continue... Nov 19, 2013 4:28:24 PM com.hazelcast.web.WebFilter INFO: Request is instance of RequestWrapper! Continue... Nov 19, 2013 4:28:24 PM com.hazelcast.web.WebFilter INFO: Request is instance of RequestWrapper! Continue... Nov 19, 2013 4:28:50 PM com.hazelcast.web.WebFilter INFO: Created new session with id: HZC9553A4C330044CA8A0C20549EE23BF0 Nov 19, 2013 4:28:50 PM com.hazelcast.web.WebFilter INFO: 197 is sessions.size and originalSessions.size: 197 Nov 19, 2013 4:28:50 PM com.hazelcast.web.WebFilter INFO: PUTTING SESSION HZC9553A4C330044CA8A0C20549EE23BF0 Nov 19, 2013 4:28:50 PM com.hazelcast.web.WebFilter INFO: Request is instance of RequestWrapper! Continue... Nov 19, 2013 4:28:50 PM com.hazelcast.web.WebFilter INFO: Request is instance of RequestWrapper! Continue... Nov 19, 2013 4:28:50 PM com.hazelcast.web.WebFilter INFO: Request is instance of RequestWrapper! Continue... Nov 19, 2013 4:28:50 PM com.hazelcast.web.WebFilter INFO: Request is instance of RequestWrapper! Continue... 10069.275: [GC [PSYoungGen: 693173K->3458K(695488K)] 877908K->188718K(2093632K), 0.0224650 secs] [Times: user=0.04 sys=0.00, real=0.02 secs] Nov 19, 2013 4:29:18 PM com.hazelcast.web.WebFilter INFO: Created new session with id: HZE46365454C2C45F98A7947AC40E404BB Nov 19, 2013 4:29:18 PM com.hazelcast.web.WebFilter INFO: 198 is sessions.size and originalSessions.size: 198 Nov 19, 2013 4:29:18 PM com.hazelcast.web.WebFilter INFO: PUTTING SESSION HZE46365454C2C45F98A7947AC40E404BB讨论(0) -
The following only works with Hazelcast 3.2-SNAPSHOT.
Lets start with the single HazelcastInstance that is shared between session replication and the application. You can do it like this:
public class HazelcastInstanceLoader { private final static ILogger logger = Logger.getLogger(HazelcastInstanceLoader.class); public static HazelcastInstance load(String instanceName) throws Exception { String configName = System.getProperty("hazelcast.config"); if (configName == null) { configName = "hazelcast.xml"; } Config config; if (configName.startsWith("file:")) { String filename = configName.substring("file:".length()); logger.info("Using hazelcast configuration file: " + filename); config = new FileSystemXmlConfig(filename); } else { logger.info("Using hazelcast classpath resource: " + configName); config = new ClasspathXmlConfig(configName); } config.setInstanceName(instanceName); return Hazelcast.getOrCreateHazelcastInstance(config); } }And now the Spring configuration:
<bean id="hazelcastInstance" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean"> <property name="staticMethod" value="com.hazelcast.webmonitor.HazelcastInstanceLoader.load"/> <property name="arguments"> <list> <value>default</value> </list> </property> </bean>So when this is loaded, we have a named HazelcastInstance 'default'.
When I configure the session replication like this:
<filter> <filter-name>hazelcast-filter</filter-name> <filter-class>com.hazelcast.web.WebFilter</filter-class> <init-param> <param-name>instance-name</param-name> <param-value>default</param-value> </init-param> </filter>Then the session filter will try to load the hazelcastinstance with that name. Since the 'default' HazelcastInstance already is created because the Spring applicationcontext is loaded, the session filter will use that instance and not create a new one.
If you need to have fully seperate configuration files, just specify a file using the 'hazelcast.conf' system property (see the code of the HazelcastInstanceLoader). Instead of relying on a system property, you could also modify the HazelcastInstanceLoader.load method to accept a 'path' property, which you can inject from Spring.
Setting different groups for different environments is a useful way to make sure that machines in these environments, don't form a group. But if you want to share configuration, and only on the details there are differences (e.g. groupnames), then you can also make use of variables in the xml:
<hazelcast> <group> <name>${groupname}</name> <password>${password}</password> </group> ... </hazelcast>Does this answer your question?
讨论(0) -
Why dont you jat put in src/main/resources a file per environement. So that you have: dev.properties,qa.properties,prod.properties
use an environement variable such as ENV. which in my situation i have put ENV=dev and then inside my spring-servlet.xml i have:
<property name="locations"> <list> <value>classpath:/${SERVER_ENV}.hazelcast.properties</value> </list> </property>and then in the same file i use properties this way:
<hz:hazelcast id="myapp"> <hz:config> <!-- Hazelcast Instance Name --> <hz:instance-name>${hz.instance.name}</hz:instance-name> <!-- Hazelcast Group Name and Password --> <hz:group name="${hz.group.name}" password="${hz.group.password}" /> <!-- Hazelcast Management Center URL --> <hz:management-center enabled="${hz.management.center.enabled}" url="${hz.management.center.url}" /> <!-- Hazelcast Tcp based network configuration --> <hz:network port="${hz.network.port}" port-auto-increment="${hz.network.port.auto.increment}"> <hz:join> <hz:multicast enabled="${hz.multicast.enabled}" multicast-group="224.2.2.3" multicast-port="54327" /> <hz:tcp-ip enabled="${hz.tcp.ip.enabled}"> <hz:members>${hz.members}</hz:members> </hz:tcp-ip> </hz:join> </hz:network> <!-- Hazelcast Distributed Map configuration --> <hz:map name="map" backup-count="${hz.map.backup.count}" max-size="${hz.map.max.size}" eviction-percentage="${hz.map.eviction.percentage}" read-backup-data="${hz.map.read.backup.data}" eviction-policy="${hz.map.eviction.policy}" merge-policy="${hz.map.merge.policy}" /> </hz:config> </hz:hazelcast>讨论(0)
加载中...
- 热议问题