RabbitMQ Error: unable to connect to nodes : nodedown

Question

I am trying to do clustering using two Rhel instances. I am able to ping each other and even when i am trying to use command empd -names, i get it is up and running on 4369.When i am use the command as sudo rabbitmqctl join_cluster rabbit@ip-10-23-20-36 i am getting the below error ensuring as well to i am stop_app first..

sudo rabbitmqctl join_cluster rabbit@ip-10-23-20-36

Clustering node 'rabbit@ip-10-23-20-36' with 'rabbit@ip-10-23-209-142' ... Error: unable to connect to nodes ['rabbit@ip-10-23-209-142']: nodedown

DIAGNOSTICS

attempted to contact: ['rabbit@ip-10-23-209-142']

rabbit@ip-10-23-209-142: * unable to connect to epmd (port 4369) on ip-10-23-209-142: nxdomain (non-existing domain)

current node details: - node name: 'rabbitmq-cli-80@ip-10-23-20-36' - home dir: /var/lib/rabbitmq - cookie hash: u7nRIpJ40Fd356iLbkDO6Q==

Things I already tried:

1. Checked the cookie name,which is same in both instances using
   sudo cat /var/lib/rabbitmq/.erlang.cookie.
2. Changed the epmd port as well export ERL_EMPD_PORT=4370
3. netstat -an |grep 4369 | grep -i listen
4. Changing the hostnames as well in GUI of plugin management.

5. Changed owner and permission also using

   sudo chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie
   sudo chmod 400 /var/lib/rabbitmq/.erlang.cookie
   

6. Add port

   sudo iptables -I INPUT -p tcp --dport 4369 --syn -j ACCEPT
   

7. sudo rabbitmqctl status

   {listeners,[{clustering,25672,"::"},{amqp,5672,"::"}]},
   

*IP-Adresses are sample adrresses.

Answer 1

As the error message says, your hostnames do not resolve:

rabbit@ip-10-23-209-142: * unable to connect to epmd (port 4369) on ip-10-23-209-142: nxdomain (non-existing domain)

You need to declare those hostnames in your DNS or in /etc/hosts on both hosts. You can then test the name resolution is working by using eg. ping(1):

ping ip-10-23-209-142 # from ip-10-23-20-36

Answer 2

I got the same error like that today and the suggestion is meaningless.

So, firstly, you should check its log at /var/log/rabbitmq/rabbitmq@[your hostname].log or you waste your time. Then you can see what's happened there.

In my case, it reported an error in file /var/db/rabbitmq/mnesia/rabbit@www/cluster_nodes.config

 Error description:


{error,{cannot_read_file,"/var/db/rabbitmq/mnesia/rabbit@www/cluster_nodes.config",
                        {1,erl_parse,["syntax error before: ","'@'"]}}}

So, I just remove this folder /var/db/rabbitmq/mnesia/rabbit@www and restart the service and it works like a charm