Kafka producer is slow on first message

问题

I have a test application in .NET that creates kafka producer and sends a few messages to a topic using Confluent client library.

For some reason the acknowledgement for the first message always arrives 1 second late, acknowledgements for subsequent messages arrive almost immediately.

Is this a normal behavior or am I missing some configuration?

I think I tried tuning all producer configs, nothing helps except setting EnableDeliveryReports = false.
Topic is not partitioned and already exists at the moment of sending the first message.

If I update the application to send messages to a few topics, then first message to each topic gets sent with 1 second delay.

static void Main()
{
    var producer = new Producer<Null, string>(new ProducerConfig
    {
        BootstrapServers = "localhost:9092",
        LingerMs = 100,
        BatchNumMessages = 1,
    });

    for (var i = 0; i < 10; i++)
    {
        var start = DateTime.Now;
        producer.ProduceAsync(
            new TopicPartition("test-topic", Partition.Any), new Message<Null, string>
            {
                Value = $"hello kafka! #{i}"
            }).Wait(2000);

        var now = DateTime.Now;
        Console.WriteLine($"{now:HH:mm:ss.fff} Message sent in {(now - start).TotalMilliseconds:N1}.");
    }

    producer.Dispose();
    Console.ReadLine();
}

Sample output:

18:06:13.605 Message sent in 1,007.0.
18:06:13.607 Message sent in 1.0.
18:06:13.608 Message sent in 1.0.
18:06:13.609 Message sent in 1.0.
18:06:13.610 Message sent in 1.0.
18:06:13.611 Message sent in 1.0.
18:06:13.612 Message sent in 1.0.
18:06:13.613 Message sent in 1.0.
18:06:13.614 Message sent in 1.0.
18:06:13.615 Message sent in 1.0.

回答1:

After some more digging in kafka request logs I found that before sending the first message producer sends API_VERSIONS and METADATA requests for all topics. After 1 second it sends again METADATA request, this time for the specific topic that producer is trying to push message to.

Setting TopicMetadataRefreshIntervalMs to something less than 1 second reduces the first message delay. As a downside producer starts sending metadata requests more often. Surprisingly setting TopicMetadataRefreshIntervalMs to something more than a second does not extend the delay.

Not sure if this is a bug or a feature.

来源：https://stackoverflow.com/questions/53728258/kafka-producer-is-slow-on-first-message

标签

.net

apache-kafka

confluent-platform