How to configure logging for Kafka producers?

Question

I am using Kafka producer client and i don\'t have any log4j configuration in my project.

On running, the program prints a lot of Kafka Debug logs which i really don\'t

Answer 1

Try adding logging.level.org.apache.kafka: DEBUG into your clients configuration properties. I am using Springboot and this is the format. Use appropriate format for your clients program.

Answer 2

I assumed you were talking about Kafka server logs. You can change log level to ERROR using following config

log4j.logger.kafka=ERROR, kafkaAppender

Hope this helps!

Answer 3

Define the logging level as below in application.yml file or your properties file.

logging:
  level:
    root: INFO
    org:
      apache:
        kafka: WARN

By defining the above, Kafka will print out only the warnings logs.

Answer 4

Use the command line flag -Dlog4j.configuration=file:/path/to/log4j.properties when running your client.

Example log4j property files:

• https://github.com/apache/kafka/blob/trunk/config/tools-log4j.properties
  • Used by tools like mirror maker, really just stdout
• https://github.com/apache/kafka/blob/trunk/config/log4j.properties
  • Used by tools like kafka server

For mirror maker and other tools that result in a call to kafka-run-class.sh, you can use the env variable KAFKA_LOG4J_OPTS (set to something like -Dlog4j.configuration=file:/path/to/log4j.properties) to change the logging configuration. See: https://github.com/apache/kafka/blob/0.10.2/bin/kafka-run-class.sh#L158

Example of my log4j.properties file for mirror maker that I use for testing.

# https://github.com/apache/kafka/blob/trunk/config/tools-log4j.properties

log4j.rootLogger=DEBUG, stderr

log4j.appender.stderr=org.apache.log4j.ConsoleAppender
log4j.appender.stderr.layout=org.apache.log4j.PatternLayout
log4j.appender.stderr.layout.ConversionPattern=[%d] %p %m (%c)%n
log4j.appender.stderr.Target=System.err

Answer 5

org.apache.log4j.Logger.getLogger("org").setLevel(Level.WARN);
org.apache.log4j.Logger.getLogger("akka").setLevel(Level.WARN);
org.apache.log4j.Logger.getLogger("kafka").setLevel(Level.WARN);