Exception during topic deletion when Kafka is hosted in Docker in Windows

Question

I host Kafka in Docker in Windows. Wurstmeister/Kafka docker image is used. Kafka data is stored in local Windows folder for persistency. Windows folder is mapped to Kafka docke

Answer 1

Delete version-2 from Zookeeper logs folder.
Delete all things in Kafka-logs folder.

Then restart the Zookeeper and Kafka server:

• zookeeper-server-start.bat D:\kafka_2.11-2.4.1\config\zookeeper.properties
• kafka-server-start.bat D:\kafka_2.11-2.4.1\config\server.properties

Answer 2

This issue still exists in Windows for Kafka ver 1.1.0 (kafka_2.12-1.1.0) when I try to delete the topic.

The topic gets marked for deletion and the Kafka server fails with java.nio.file.AccessDeniedException when trying to rename the logs directory 'test-0'

Deleting the whole test-0 logs folder does not help. Reinstalling the Kafka server does not help either - even after reinstalling, the info about the topic marked for deletion remains.

Took me a couple of hours to figure out that the info about the topic sits in the Zookeeper - in one of the log files!

Solution

Stop the Zookeeper process. Go to your Zookeeper logs folder zookeeper-3.x.x\bin\zookeeper-3.x.xdata\version-2\ and delete the latest log.xx files. Restart Zookeper. Restart Kafka server.

Answer 3

Deletion of topic fails due to Java's File.rename function. It works differently in some cases in Windows environment (for example if file is in use). Kafka developers already changed this function to Utils.atomicMoveWithFallback (see this issue for details), but seems it was not included into Kafka 2.11-0.11.0. So you need to work with Kafka version that has this fix. Hope this will helps.

Answer 4

This solved the problem
1.Stop both Kafka and Zookeeper processes.
2.Delete all old log directories.
3.Change the log.dir to point to new directory in server.properties
4.Change the dataDir to point to new directory in zookeeper.properties

Then restart the Zookeeper and Kafka server:

C:\kafka_2.12-2.4.0> zookeeper-server-start.bat .\config\zookeeper.properties
C:\kafka_2.12-2.4.0> kafka-server-start.bat .\config\server.properties

Answer 5

I solved this problem after reseting the credentials for my shared drive

Docker config > Shared Drives > Reset credentials

Answer 6

1, Change the log.dir with a new name in server.properties ->Kafka/config

log.dir=C:/Programs/kafka/kafka_2.12-2.3.0/kafka-test-logs

2, Remove the old log folder from C:/Programs/kafka/kafka_2.12-2.3.0/

3, Remove all logs and snapshot from C:\Programs\zookeeper\apache-zookeeper-3.5.5-bin\data or delete the data folder where your log is stored

Additionally, I had an error when starting the consumer(Leader Not Available Kafka in Console Producer),

I added,

port = 9092 advertised.host.name = localhost

to the server.properties

Now am able to publish and consume messages