Kafka + Zookeeper: Connection to node -1 could not be established. Broker may not be available

Question

I am running in my locahost both Zookeeper and Kafka (1 instance each).

I create succesfully a topic from kafka:

./bin/kafka-topics.sh --zookeeper lo         


        

Answer 1

I found the error. Observing zookeeper logs when the server started I noticed:

server.1=mylocal-0.:2888:3888

with a dot (.) after the name of the host.

The script that produces the zookeeper's config is from https://github.com/kubernetes/contrib/blob/master/statefulsets/zookeeper/zkGenConfig.sh

Looking inside I see that DOMAIN is not filled:

HOST=`hostname -s`
DOMAIN=`hostname -d`

function print_servers() {
    for (( i=1; i<=$ZK_REPLICAS; i++ ))
    do
        echo "server.$i=$NAME-$((i-1)).$DOMAIN:$ZK_SERVER_PORT:$ZK_ELECTION_PORT"
    done
}

For my case (localhost) I don't need domain, so I removed that variable.

Now zookeeper and kafka communicate with no errors.

Answer 2

Change:

#listeners=PLAINTEXT://:9092`

in server.properties to:

listeners=PLAINTEXT://localhost:9092

Note: You also need to uncomment this statement aka remove the # symbol.

Answer 3

If this happens suddenly after it was working, you should try to restart Kafka first.

In my case, restarting solved the problem:

$docker-compose down && docker-compose up -d

Answer 4

If you want to set up it for local then you need to un comment the below line in path_to_kafka_folder\kafka_2.13-2.6.0\config\server.properties

listeners=PLAINTEXT://localhost:9092

Answer 5

UPD: if you are running in single-node mode:

I have seen this message in spark console log while trying to deploy application. Solved by changing this parameter in server.properties:

listeners=PLAINTEXT://myhostname:9092

to

listeners=PLAINTEXT://localhost:9092

make sure that you have java process listening on 9092 with netstat -lptu