1. 技术文章
  2. Apache ActiveMq Artemis client reconnecting to next available broker in clustered HA replication/shared-data store

Apache ActiveMq Artemis client reconnecting to next available broker in clustered HA replication/shared-data store

由 ε祈祈猫儿з 提交于 2021-02-05 11:40:29

问题


Broker.xml (host1) and host2 just the port number changes to 61616 and slave as configuration. In reference with Apache Artemis client fail over discovery

<?xml version="1.0" encoding="UTF-8" standalone="no"?>

<configuration xmlns="urn:activemq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">
   <core xmlns="urn:activemq:core">

      <bindings-directory>./data/bindings</bindings-directory>

      <journal-directory>./data/journal</journal-directory>

      <large-messages-directory>./data/largemessages</large-messages-directory>

      <paging-directory>./data/paging</paging-directory>
      <!-- Connectors -->

      <connectors>
         <connector name="netty-connector">tcp://10.64.60.100:61617</connector><!-- direct ip addres of host myhost1 -->
         <connector name="broker2-connector">tcp://myhost2:61616</connector> <!-- ip 10.64.60.101 <- mocked up ip for security reasons -->
      </connectors>

      <!-- Acceptors -->
      <acceptors>
         <acceptor name="amqp">tcp://0.0.0.0:61617?amqpIdleTimeout=0;tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=CORE,AMQP;useEpoll=true</acceptor>
      </acceptors>

      <ha-policy>
       <replication>
          <master/>
       </replication>
    </ha-policy>

      <cluster-connections>
         <cluster-connection name="myhost1-cluster">
            <connector-ref>netty-connector</connector-ref>
            <retry-interval>500</retry-interval>
            <use-duplicate-detection>true</use-duplicate-detection>
            <message-load-balancing>ON_DEMAND</message-load-balancing>
            <max-hops>1</max-hops>
              <static-connectors>
             <connector-ref>broker2-connector</connector-ref> <!-- defined in the connectors -->
            </static-connectors>
         </cluster-connection>
      </cluster-connections>

      <security-settings>
         <security-setting match="#">
            <permission type="createNonDurableQueue" roles="amq"/>
            <permission type="deleteNonDurableQueue" roles="amq"/>
            <permission type="createDurableQueue" roles="amq"/>
            <permission type="deleteDurableQueue" roles="amq"/>
            <permission type="createAddress" roles="amq"/>
            <permission type="deleteAddress" roles="amq"/>
            <permission type="consume" roles="amq"/>
            <permission type="browse" roles="amq"/>
            <permission type="send" roles="amq"/>
            <permission type="manage" roles="amq"/>
         </security-setting>
      </security-settings>

      <address-settings>
         <!--default for catch all-->
         <address-setting match="#">
            <dead-letter-address>DLQ</dead-letter-address>
            <expiry-address>ExpiryQueue</expiry-address>
            <redelivery-delay>0</redelivery-delay>
            <!-- with -1 only the global-max-size is in use for limiting -->
            <max-size-bytes>-1</max-size-bytes>
            <message-counter-history-day-limit>10</message-counter-history-day-limit>
            <address-full-policy>PAGE</address-full-policy>
            <auto-create-queues>true</auto-create-queues>
            <auto-create-addresses>true</auto-create-addresses>
             <auto-delete-queues>false</auto-delete-queues>
            <auto-delete-created-queues>false</auto-delete-created-queues>
            <auto-delete-addresses>false</auto-delete-addresses>
        </address-setting>
      </address-settings>
   </core>
</configuration>

Client which uses Java and producer template to push message to direct endpoint and then routed to queue produces message:

say Java-camel-producer-client

    package com.demo.artemis;

    import org.apache.camel.CamelContext;
    import org.apache.camel.ProducerTemplate;
    import org.apache.camel.spring.SpringCamelContext;
    import org.springframework.context.ApplicationContext;
    import org.springframework.context.support.ClassPathXmlApplicationContext;

    public class ToRoute {

         public static void main(String[] args) throws Exception { 
                ApplicationContext appContext = new ClassPathXmlApplicationContext(
                        "camel-context-producer.xml");
                ProducerTemplate template = null;
                CamelContext camelContext = SpringCamelContext.springCamelContext(
                        appContext, false);
                try {            
                    camelContext.start();
                     template = camelContext.createProducerTemplate();
                    String msg = null;
                    int loop =0;
                    int i = 0;
                    while(true) {
                        if (i%10000==0) {
                            i=1;
                            loop=loop+1;
                        }
                        if(loop==2) break;
                        msg ="---> "+(++i);             
                        template.sendBody("direct:toDWQueue", msg);
                    }
                } finally {
                    if (template != null) {
                        template.stop();
                    }
                    camelContext.stop();
                }
         }
    }

Camel Context which sends message to the Queue: Say camel-producer-client

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" 
    xmlns:aop="http://www.springframework.org/schema/aop" 
    xmlns:context="http://www.springframework.org/schema/context" 
    xmlns:jee="http://www.springframework.org/schema/jee" 
    xmlns:tx="http://www.springframework.org/schema/tx" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/aop 
        http://www.springframework.org/schema/aop/spring-aop-3.1.xsd
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context-3.1.xsd
        http://www.springframework.org/schema/jee
        http://www.springframework.org/schema/jee/spring-jee-3.1.xsd
        http://www.springframework.org/schema/tx
        http://www.springframework.org/schema/tx/spring-tx-3.1.xsd
        http://camel.apache.org/schema/spring
        http://camel.apache.org/schema/spring/camel-spring.xsd">
     <bean id="jmsConnectionFactory" class="org.apache.activemq.artemis.jms.client.ActiveMQJMSConnectionFactory">
      <constructor-arg index="0" value="(tcp://myhost:61616,tcp://myhost1:61617)?ha=true;reconnectAttempts=-1;"/>
    </bean>

    <bean id="jmsPooledConnectionFactory" class="org.messaginghub.pooled.jms.JmsPoolConnectionFactory" init-method="start" destroy-method="stop">
      <property name="maxConnections" value="10" />
      <property name="connectionFactory" ref="jmsConnectionFactory" />
    </bean>

    <bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
      <property name="connectionFactory" ref="jmsPooledConnectionFactory" />
      <property name="concurrentConsumers" value="5" />
    </bean>

    <bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
      <property name="configuration" ref="jmsConfig" />
      <property name="streamMessageTypeEnabled" value="true"/>
    </bean>

        <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
      <endpoint id="myqueue" uri="jms:queue:myExampleQueue" />

      <route>
        <from uri="direct:toMyExample"/>
       <transform>
      <simple>MSG FRM DIRECT TO MyExampleQueue : ${bodyAs(String)}</simple>
        </transform>
        <to uri="ref:myqueue"/>
      </route>
    </camelContext>

  </beans>

Camel consumer which reads from MyExampleQueue and transforms to OutboundQueue: (This context file is started using camel “org.apache.camel.spring.Main –ac camel-consumer-client

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" 
    xmlns:aop="http://www.springframework.org/schema/aop" 
    xmlns:context="http://www.springframework.org/schema/context" 
    xmlns:jee="http://www.springframework.org/schema/jee" 
    xmlns:tx="http://www.springframework.org/schema/tx" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.springframework.org/schema/aop 
        http://www.springframework.org/schema/aop/spring-aop-3.1.xsd
        http://www.springframework.org/schema/beans
        http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
        http://www.springframework.org/schema/context
        http://www.springframework.org/schema/context/spring-context-3.1.xsd
        http://www.springframework.org/schema/jee
        http://www.springframework.org/schema/jee/spring-jee-3.1.xsd
        http://www.springframework.org/schema/tx
        http://www.springframework.org/schema/tx/spring-tx-3.1.xsd
        http://camel.apache.org/schema/spring
        http://camel.apache.org/schema/spring/camel-spring.xsd">

        <bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
            <property name="environment">
                <props>
                    <prop key="java.naming.factory.initial">org.apache.activemq.artemis.jndi.ActiveMQInitialContextFactory</prop>
                                <prop key="connectionFactory.ConnectionFactory">(tcp://myhost1:61617,tcp://myhost2:61616)?ha=true;retryInterval=1000;retryIntervalMultiplier=1.0;reconnectAttempts=-1;</prop>
                                  <prop key="queue.queue/MyExampleQueue">MyExampleQueue</prop>
                    <prop key="queue.queue/OutBoundQueue">OutBoundQueue</prop>
                </props>
            </property>
        </bean>
           <bean id="jndiFactoryBean" class="org.springframework.jndi.JndiObjectFactoryBean">
            <property name="jndiName" value="ConnectionFactory"/> 
            <property name="jndiTemplate" ref="jndiTemplate"/>
              <property name="cache" value="true"/>     
        </bean> 

        <bean id="jndiDestinationResolver" class="org.springframework.jms.support.destination.JndiDestinationResolver">
            <property name="jndiTemplate" ref="jndiTemplate"/>
              <property name="cache" value="true"/>
    <!-- dynamic destination if the destination name  is not found in JNDI -->
             <property name="fallbackToDynamicDestination" value="true"/>
        </bean>

       <bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
            <property name="connectionFactory" ref="jndiFactoryBean"/>
            <property name="destinationResolver" ref="jndiDestinationResolver"/>
                 <property name="concurrentConsumers" value="10" />
        </bean>

        <bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
               <property name="configuration" ref="jmsConfig" />
        </bean>

        <camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
      <endpoint id="inqueue" uri="jms:queue:MyExampleQueue" />
      <endpoint id="outqueue" uri="jms:queue:OutBoundQueue" />

      <route>
        <from uri="ref:inqueue" />
        <convertBodyTo type="java.lang.String" />
        <transform>
          <simple>MSG FRM MyExampleQueue TO OutboundQueue : ${bodyAs(String)}</simple>
        </transform>
        <to uri="ref:outqueue" />
      </route>
    </camelContext>
  </beans>

I think the issue i am facing is similar to this ticket http://activemq.2283324.n4.nabble.com/Connection-to-the-backup-after-master-shutdown-td4753770.html

When using the Camel context, i see the client consumer is redirected to slave when the master is not available (i was using the JNDI with JMSPoolConnectionFactory).

The Java-producer-client sends the message using the producer template to direct endpoint which is transformed and routed to queue. When the master is down, the client was not able to fail-over and connect to slave. (Question: Is this the behavior expected when the broker are in HA mode).

After reconnect attempt i see below exception When the master is down.

Caused by: javax.jms.JMSException: AMQ219016: Connection failure detected. Unblocking a blocking call that will never get a response

on the other hand camel-context-consumer which was started using Main, was able to fail-over to slave automatically. In the console i did notice that the consumer count 10 was available in slave host and processing data. But when the master is back up, the client didn't switch over to the live Master node. Question :Is this also expected?

For creating the connectionfactory below conventions is used. 

(tcp://myhost:61616,tcp://myhost1:61617)?ha=true;reconnectAttempts=-1;

Question: Do we need to configure the broacast and discovery group, even when using a static connector?


回答1:


The error stating "Unblocking a blocking call that will never get a response" is expected if failover happens when the client is in the middle of a blocking call (e.g. sending a durable message and waiting for an ack from the broker, committing a transaction, etc.). This is discussed further in the documentation.

The fact that clients don't switch back to the master broker when it comes back is also expected given your configuration. In short, you haven't configured failback properly. Your master should have:

<ha-policy>
   <replication>
      <master>
         <check-for-live-server>true</check-for-live-server>
      </master>
   </replication>
</ha-policy>

And your slave should have:

<ha-policy>
   <replication>
      <slave>
         <allow-failback>true</allow-failback>
      </slave>
   </replication>
</ha-policy>

This is also discussed in the documentation.

Lastly, you do not need to configure the broadcast and discovery groups when using a static connector.




回答2:


After handling the JMSException in the producer (also was able to fail-over successfully)

  <!--added this handler-->
    <bean id="deadLetterErrorHandler"
        class="org.apache.camel.builder.DeadLetterChannelBuilder">
        <property name="deadLetterUri" value="log:dead" />
    </bean>

<!-- referred the handler -->
<route errorHandlerRef="deadLetterErrorHandler">
      <from uri="direct:toMyQueue" />
            <transform>
                <simple>MSG FRM DIRECT TO MyExampleQueue : ${bodyAs(String)}
                </simple>
            </transform>
            <to uri="ref:myqueue" />
</route>

spun up two VM and used 172.28.128.28 (master) and 172.28.128.100 (was slave/backup) From below logs, when the master is was down the slave started at broker side and client failover logs are below.

2020-06-06 07:26:11 DEBUG NettyConnector:1259 - NettyConnector [host=172.28.128.28, port=61616, httpEnabled=false, httpUpgradeEnabled=false, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] host 1: 172.28.128.100 ip address: 172.28.128.100 host 2: 172.28.128.28 ip address: 172.28.128.28
2020-06-06 07:26:11 DEBUG ClientSessionFactoryImpl:272 - Setting up backup config = TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=172-28-128-100 for live = TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=172-28-128-28
2020-06-06 07:26:11 DEBUG JmsConfiguration$CamelJmsTemplate:502 - Executing callback on JMS Session: JmsPoolSession { ActiveMQSession->ClientSessionImpl [name=ab4faacf-a801-11ea-9c64-0a002700000c, username=null, closed=false, factory = org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl@7b18658a, metaData=(jms-session=,)]@1118d539 }
2020-06-06 07:26:11 DEBUG JmsConfiguration:621 - Sending JMS message to: ActiveMQQueue[myExampleQueue] with message: ActiveMQMessage[null]:PERSISTENT/ClientMessageImpl[messageID=0, durable=true, address=null,userID=null,properties=TypedProperties[]]
2020-06-06 07:26:11 DEBUG DefaultProducerCache:169 - >>>> direct://toMyQueue Exchange[]
2020-06-06 07:26:11 DEBUG SendProcessor:167 - >>>> ref://myqueue Exchange[]
...
2020-06-06 07:27:15 DEBUG ClientSessionFactoryImpl:800 - Trying reconnection attempt 0/1
2020-06-06 07:27:15 DEBUG ClientSessionFactoryImpl:1102 - Trying to connect with connectorFactory = org.apache.activemq.artemis.core.remoting.impl.netty.NettyConnectorFactory@2fd9fb34, connectorConfig=TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=172-28-128-100
2020-06-06 07:27:15 DEBUG NettyConnector:508 - Connector + NettyConnector [host=172.28.128.100, port=61616, httpEnabled=false, httpUpgradeEnabled=false, useServlet=false, servletPath=/messaging/ActiveMQServlet, sslEnabled=false, useNio=true] using nio
2020-06-06 07:27:15 DEBUG client:670 - AMQ211002: Started NIO Netty Connector version 4.1.48.Final to 172.28.128.100:61616
2020-06-06 07:27:15 DEBUG NettyConnector:805 - Remote destination: /172.28.128.100:61616
2020-06-06 07:27:15 DEBUG NettyConnector:661 - Added ActiveMQClientChannelHandler to Channel with id = 62e089d3 
2020-06-06 07:27:15 DEBUG ClientSessionFactoryImpl:809 - Reconnection successful
2020-06-06 07:27:15 DEBUG ClientSessionFactoryImpl:277 - ClientSessionFactoryImpl received backup update for live/backup pair = TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=172-28-128-100 / null but it didn't belong to TransportConfiguration(name=netty-connector, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=172-28-128-100
2020-06-06 07:27:15 DEBUG JmsConfiguration$CamelJmsTemplate:502 - Executing callback on JMS Session: JmsPoolSession { ActiveMQSession->ClientSessionImpl [name=d200c1a0-a801-11ea-9c64-0a002700000c, username=null, closed=false, factory = org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl@12abcd1e, metaData=(jms-session=,)]@33d28f0a }
2020-06-06 07:27:16 DEBUG JmsConfiguration:621 - Sending JMS message to: ActiveMQQueue[myExampleQueue] with message: ActiveMQMessage[null]:PERSISTENT/ClientMessageImpl[messageID=0, durable=true, address=null,userID=null,properties=TypedProperties[]]
2020-06-06 07:27:16 DEBUG DefaultProducerCache:169 - >>>> direct://toMyQueue Exchange[]
2020-06-06 07:27:16 DEBUG SendProcessor:167 - >>>> ref://myqueue Exchange[]

When the client (consumer) using camel routes within the broker (queue to queue route), or broker (queue) to spring bean the JMS Exception didn't occur, but handling and redelivering those would be helpful.



来源：https://stackoverflow.com/questions/62218408/apache-activemq-artemis-client-reconnecting-to-next-available-broker-in-clustere

标签
java
apache-camel
activemq-artemis
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!