kafka-producer-api

问题 I would like to have Exactly-once semantics, but I don't want to read message with Consumer. I'd rather read messages with Kafka Streams AP. If I add processing.guarantee=exactly_once to Stream configuration, will exactly-once semantics be kept? 回答1: Exactly-once processing is based on a read-process-write pattern. Kafka Streams uses this pattern and thus, if you write a regular Kafka Streams application that writes the result back to a Kafka topic, you will get exactly-once processing

问题 AFAIK, max.poll.interval.ms is introduced in Kafka 0.10.1. However it is still unclear that when we can use both session.timeout.ms and max.poll.interval.ms Consider the use casein which heartbeat thread is not responding, but my processing thread as it has higher value set, it still is processing the record. But as heartbeat thread is down then after crossing session.timeout.ms, what exactly happens. Because I've observed in POC that consumer re-balance doesn't happen until it reaches max

问题 AFAIK, max.poll.interval.ms is introduced in Kafka 0.10.1. However it is still unclear that when we can use both session.timeout.ms and max.poll.interval.ms Consider the use casein which heartbeat thread is not responding, but my processing thread as it has higher value set, it still is processing the record. But as heartbeat thread is down then after crossing session.timeout.ms, what exactly happens. Because I've observed in POC that consumer re-balance doesn't happen until it reaches max

问题 Please help I'm wondering why the kafka producer always connect to the localhost however there the broker ip is not the localhost. So, is there any help ? any ideas ? import org.apache.kafka.clients.producer.KafkaProducer; import org.apache.kafka.clients.producer.ProducerConfig; import org.apache.kafka.clients.producer.ProducerRecord; import org.apache.kafka.common.serialization.StringSerializer; import java.util.Properties; public class ProducerDemo { public static void main(String[] args)

问题 Please help I'm wondering why the kafka producer always connect to the localhost however there the broker ip is not the localhost. So, is there any help ? any ideas ? import org.apache.kafka.clients.producer.KafkaProducer; import org.apache.kafka.clients.producer.ProducerConfig; import org.apache.kafka.clients.producer.ProducerRecord; import org.apache.kafka.common.serialization.StringSerializer; import java.util.Properties; public class ProducerDemo { public static void main(String[] args)

问题 Please help I'm wondering why the kafka producer always connect to the localhost however there the broker ip is not the localhost. So, is there any help ? any ideas ? import org.apache.kafka.clients.producer.KafkaProducer; import org.apache.kafka.clients.producer.ProducerConfig; import org.apache.kafka.clients.producer.ProducerRecord; import org.apache.kafka.common.serialization.StringSerializer; import java.util.Properties; public class ProducerDemo { public static void main(String[] args)

问题 I am using Spring Kafka template for producing messages. And the rate at which it is producing the messages is too slow. Takes around 8 mins for producing 15000 messages. Following is How I created the Kafka template: @Bean public ProducerFactory<String, GenericRecord> highSpeedAvroProducerFactory( @Qualifier("highSpeedProducerProperties") KafkaProperties properties) { final Map<String, Object> kafkaPropertiesMap = properties.getKafkaPropertiesMap(); System.out.println(kafkaPropertiesMap);

问题 I'm using a non Blocking (Async) sending message to Kafka using this : ListenableFuture<SendResult<Integer, String>> future = template.send(record); future.addCallback(new ListenableFutureCallback<SendResult<Integer, String>>() { @Override public void onSuccess(SendResult<Integer, String> result) { handleSuccess(data); } @Override public void onFailure(Throwable ex) { handleFailure(data, record, ex); } }); This work perfectly when the send action does its work. But when there is a connection

问题 I'm working on Kafka 0.9. I'm wondering if there is any approach to retrieve a message, which has been processed, from its topic by knowing the partition and offset. For example, the consumer is currently consuming the message at partition 1 and offset 10. And I want to get the message at the same partition and offset 5. One way that I can think of is to reset the offset to 5 and consume one single message. But the poll() method can only return a batch of messages. So I have to take the first

问题 I'm working on Kafka 0.9. I'm wondering if there is any approach to retrieve a message, which has been processed, from its topic by knowing the partition and offset. For example, the consumer is currently consuming the message at partition 1 and offset 10. And I want to get the message at the same partition and offset 5. One way that I can think of is to reset the offset to 5 and consume one single message. But the poll() method can only return a batch of messages. So I have to take the first