kafka-producer-api

问题 Whenever I am trying to read the message from kafka queue, I am getting following exception : [error] (run-main-0) java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to com.harmeetsingh13.java.Customer java.lang.ClassCastException: org.apache.avro.generic.GenericData$Record cannot be cast to com.harmeetsingh13.java.Customer at com.harmeetsingh13.java.consumers.avrodesrializer.AvroSpecificDeserializer.infiniteConsumer(AvroSpecificDeserializer.java:79) at

问题 I am new to Kafka and working with new KafkaProducer and KafkaConsumer, version : 0.9.0.1 Is there any way in java to alter/update the number of partitions for a specific topic after it has been created. I am not using zookeeper to create topic. My KafkaProducer is automatically creating topics when publish request arrives. I can also provide more details if these are not enough 回答1: Yes, it's possible. You have to access the AdminUtils scala class in kafka_2.11-0.9.0.1.jar to add partitions.

问题 I'm new to Kafka and working on a prototype to connect a proprietary streaming service into Kafka. I'm looking to get the key of the last message sent on a topic as our in-house stream consumer needs to logon with the ID of the last message it received when connecting. Is it possible, using either the KafkaProducer or a KafkaConsumer to do this? I've attempted to do the following using a Consumer, but when also running the console consumer I see messages replayed. // Poll so we know we're

问题 I am fairly new to kafka so forgive me if this question is trivial. I have a very simple setup for purposes of timing tests as follows: Machine A -> writes to topic 1 (Broker) -> Machine B reads from topic 1 Machine B -> writes message just read to topic 2 (Broker) -> Machine A reads from topic 2 Now I am sending messages of roughly 1400 bytes in an infinite loop filling up the space on my small broker very quickly. I'm experimenting with setting different values for log.retention.ms, log

问题 I am using Kafka v0.10.1.1 with Spring-boot . I am trying to produce a message in a Kafka topic mobile-user using the below producer code: Topic mobile-user have 5 partitions and 2 replication factor . I have attached my Kafka settings at the end of the question . package com.service; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.kafka.core.KafkaTemplate; import org.springframework.kafka

问题 I want to read Apache logs in kafka and then further process in to Spark Streaming.I am new to kafka. As far as I have understand I have to write a producer class to read logs file. 回答1: You can do so by creating a connector which sources each line of the log file into the Kafka topic. Check out the example here: https://docs.confluent.io/current/connect/devguide.html#connect-developing-simple-connector 来源： https://stackoverflow.com/questions/46508901/how-to-read-logs-from-file-in-kafka

问题 This is a continuation of my previous question I was experimenting kafka's min.insync.replicas and here's the summary: Setup 3 brokers in local, created a topic insync with min.insync.replicas=2 . Messages were produced by kafka-console-producer with acks=all and read by kafka-console-consumer Bought down 2 brokers leaving just 1 insync.replicas and was expecting an exception in producer as mentioned here and here But it never happened and producer was producing messages and consumer was

问题 I am trying to setup a kafka producer with KafkaAvroSerialzer for value. And I am facing this error wheneve rit is trying to created the Producer. I am using all the jars provided in confluent 5.2.1 java.lang.NoClassDefFoundError: Could not initialize class io.confluent.kafka.schemaregistry.client.rest.RestService at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.<init>(CachedSchemaRegistryClient.java:104) at io.confluent.kafka.schemaregistry.client

问题 I am new to kafka. When I run this command javac -cp "C:\kafka\kafka_2.11-0.10.2.0\libs\kafka-clients-0.10.2.0.jar" *.java I get an error message error: package org.apache.kafka.clients.producer does not exist 回答1: In addition to the answer of @Abhishek:- First download the Kafka-clients library dependencies from [https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients] Copy the org folder and paste it in the lib folder of our main Kafka directory. Then Run the above two given

问题 I'm not able to use Kafka producer in Android app. I'm getting the below exception., Caused by: java.lang.ClassNotFoundException: Didn't find class "javax.management.DynamicMBean" on path: DexPathList[[dex file "/data/data/com.a.kafkaproducer/files/instant-run/dex/slice-zookeeper-3.4.6_8a38f89467d40a18afbd8baad7b29088157e6083-classes.dex", dex file "/data/data/com.a.kafkaproducer/files/instant-run/dex/slice-zkclient-0.8_e811ffc59b2b86f143b12056ea3479df3f628d5a-classes.dex", dex file "/data