kafka-producer-api

问题 Below is the config of job in etl.xml <batch:job id="procuerJob"> <batch:step id="Produce"> <batch:partition partitioner="partitioner"> <batch:handler grid-size="${ partitioner.limit}"></batch:handler> <batch:step> <batch:tasklet> <batch:chunk reader="Reader" writer="kafkaProducer" commit-interval="20000"> </batch:chunk> <batch:listeners> <batch:listener ref="producingListener" /> </batch:listeners> </batch:tasklet> </batch:step> </batch:partition> </batch:step> </batch:job> below is the code

问题 While producing AVRO data to Kafka, Avro serializer writing the same schema ID in the byte array which is used while writing the data. Kafka Consumer fetches the schema from Schema Registry based on schema ID in the byte array received. So same schema ID is used in both i.e. Producer and Consumer and so the schema. But why many article including this one says The consumer's schema could differ from the producer's. Please help me in understanding this. 回答1: Kafka Consumer fetches the schema

问题 While trying timestamp in ProducerRecord; I found something weird. After sending few messages from the producer, I ran kafka-console-consumer.sh and verified that those messages are in the topic. I stopped the producer and waited for a minute. When I reran kafka-console-consumer.sh then it did not show the messages that I generated previously. I also added producer.flush() and producer.close() but the outcome was still the same. Now, when I stopped using timestamp field then everything worked

问题 This is my config: props.put(ProducerConfig.ACKS_CONFIG, "all"); props.put(ProducerConfig.RETRIES_CONFIG, "1"); props.put(ProducerConfig.LINGER_MS_CONFIG, "1"); and try { producer.send(record); } catch (Throwable ex) { log.error(ex, "exception."); } but we found message missing. if the network dithering will lead this? do we need to send with Callback? producer.send(record, new Callback() { @Override public void onCompletion(RecordMetadata metadata, Exception exception) {} }) 回答1: A Kafka

问题 I'm getting the error: org.apache.kafka.common.errors.TimeoutException: Topic testtopic2 not present in metadata after 60000 ms. When trying to produce to the topic in my local kafka instance on windows using Java. Note that the topic testtopic2 exists and I'm able produce messages to it using the windows console producer just fine. Below the code that I'm using: import java.util.Properties; import org.apache.kafka.clients.CommonClientConfigs; import org.apache.kafka.clients.producer.Callback

问题 We are sending message with headers to Kafka using org.apache.kafka.clients.producer.ProducerRecord public ProducerRecord(String topic, Integer partition, K key, V value, Iterable<Header> headers) { this(topic, partition, (Long)null, key, value, headers); } How can I actually see these headers using command. kafka-console-consumer.sh only shows me payload and no headers. 回答1: You can use the excellent kafkacat tool. Sample command: kafkacat -b kafka-broker:9092 -t my_topic_name -C \ -f '\nKey

问题 We are sending message with headers to Kafka using org.apache.kafka.clients.producer.ProducerRecord public ProducerRecord(String topic, Integer partition, K key, V value, Iterable<Header> headers) { this(topic, partition, (Long)null, key, value, headers); } How can I actually see these headers using command. kafka-console-consumer.sh only shows me payload and no headers. 回答1: You can use the excellent kafkacat tool. Sample command: kafkacat -b kafka-broker:9092 -t my_topic_name -C \ -f '\nKey

来源： https://stackoverflow.com/questions/57666189/unable-to-read-avro-messages-using-kafka-avro-console-consumer-serializationexc

来源： https://stackoverflow.com/questions/57666189/unable-to-read-avro-messages-using-kafka-avro-console-consumer-serializationexc

来源： https://stackoverflow.com/questions/58753250/apache-kafka-the-order-of-messages-in-partition-guarantee