KafkaTimeoutError('Failed to update metadata after 60.0 secs.')

Question

I am writing a Kafka producer using Python 3.6,the Python-kafka client version is 1.4.4。The Kafka version is: 2.1.0 & 1.1.1(two version are tried),but when I write a message to producer,throw this error:

KafkaTimeoutError('Failed to update metadata after 60.0 secs.')

This is my client code:

producer = KafkaProducer(
  bootstrap_servers=['mq-server:9092'],
  api_version = (0,10,2,0) # solve no broker error
)

producer.send("dolphin-test".encode('utf-8'),b"test")

This is server config I am modified:

listeners=PLAINTEXT://10.142.0.2:9092
advertised.listeners=PLAINTEXT://10.142.0.2:9092

When using script to produce and consume message,it works fine!This is the client trace output:

D:\project\souce\pydolphin-service>D:/Programs/Python/Python37/python.exe d:/project/souce/pydolphin-service/dolphin/producer.py
Traceback (most recent call last):
  File "d:/project/souce/pydolphin-service/dolphin/producer.py", line 14, in <module>
    future = producer.send('my-topic', b'raw_bytes')
  File "D:\Programs\Python\Python37\lib\site-packages\kafka\producer\kafka.py", line 555, in send
    self._wait_on_metadata(topic, self.config['max_block_ms'] / 1000.0)
  File "D:\Programs\Python\Python37\lib\site-packages\kafka\producer\kafka.py", line 682, in _wait_on_metadata
    "Failed to update metadata after %.1f secs." % (max_wait,))
kafka.errors.KafkaTimeoutError: KafkaTimeoutError: Failed to update metadata after 60.0 secs.

I am aready searching from google & baidu & yandex.

Answer 1

Is there a firewall b/w your dev box and kafka servers .Can you try telnet mq-server 9092

Answer 2

Tweak the advertised config:

advertised.listeners=PLAINTEXT://external-ip:9092

Modify the NAT internal network ip to public network ip. Because the producer would send message to this address. Producer only could access the public up.