How To create a kafka topic from java for KAFKA-2.1.1-1.2.1.1?

Deadly 提交于 2019-12-11 16:56:59

问题


I am working on java interface which would take user input of topic name, replication and partition to create a kafka topic in KAFKA-2.1.1-1.2.1.1. This is code that I have used from other sources but it seems to be for previous version of kafka

import kafka.admin.AdminOperationException;
import org.I0Itec.zkclient.ZkClient;
import org.I0Itec.zkclient.ZkConnection;

import java.util.Properties;
import java.util.concurrent.TimeUnit;

import kafka.admin.AdminUtils;
import kafka.utils.ZKStringSerializer$;
import kafka.utils.ZkUtils;

public class CreateTopic {
private String topicName;
private int partitions;
private int replication;
   final static String zookeeperConnect = System.getenv("ZOOKEEPER_CONNECT");
final static String zookeeperConnect = "bdlkafdev01.isus.emc.com:2181"; 
int nRetries = 3;
 boolean resetTopic;





public CreateTopic(String TP,int par,int rep,boolean RT){
    this.topicName=TP;
    this.partitions=par;
    this.replication=rep;
    this.resetTopic=RT;


}



  public void Intiate() throws Exception {
if (topicName.length() < 1) {
  throw new Exception("Missing environment variable 'TOPIC_NAME'!");
}

final int sessionTimeoutMs = 10 * 1000;
final int connectionTimeoutMs = 8 * 1000;
// Note: You must initialize the ZkClient with ZKStringSerializer.  If you don't, then
// createTopic() will only seem to work (it will return without error).  The topic will exist in
// only ZooKeeper and will be returned when listing topics, but Kafka itself does not create the
// topic.
ZkClient zkClient = new ZkClient(
    zookeeperConnect,
    sessionTimeoutMs,
    connectionTimeoutMs,
    ZKStringSerializer$.MODULE$);

// Security for Kafka was added in Kafka 0.9.0.0
final boolean isSecureKafkaCluster = false;
ZkUtils zkUtils = new ZkUtils(zkClient, new ZkConnection(zookeeperConnect), true);

if (resetTopic && AdminUtils.topicExists(zkUtils, topicName)) {
  System.out.println("Deleting topic " + topicName);
  AdminUtils.deleteTopic(zkUtils, topicName);

  while (AdminUtils.topicExists(zkUtils, topicName)) {
    System.out.println("Waiting for kafka to really delete topic ...");
    TimeUnit.SECONDS.sleep(1);
  }
}

if (!AdminUtils.topicExists(zkUtils, topicName)) {
  tryCreate(zkUtils, topicName, nRetries);
} else {
  System.out.println("Topic \"" + topicName + "\" already exists! Nothing to do here.");
}

zkClient.close();
  }

  private void tryCreate(ZkUtils zkUtils, String topicName, int nRetriesLeft) throws InterruptedException {
System.out.println("Creating topic " + topicName);
Properties topicConfig = new Properties(); // add per-topic configurations settings here
try {
  AdminUtils.createTopic(zkUtils, topicName, partitions, replication, topicConfig);
} catch (Exception e) {
  if (nRetriesLeft <= 0) {
    throw new RuntimeException("Failed to create topic \"" + topicName + "\". Is Kafka and Zookeeper running?");
  } else {
    System.out.println("Failed to create topic, trying again in 5 seconds...");
    TimeUnit.SECONDS.sleep(5);
    tryCreate(zkUtils, topicName, nRetriesLeft - 1);
  }
}

}

So here the problem is where do I add the authentication detail as I have security enabled in Kafka. Plus is there something wrong with the code?

PS:this is the first time I am working on this so if any resources which could help me understand this better would also be of great help.

来源:https://stackoverflow.com/questions/47387495/how-to-create-a-kafka-topic-from-java-for-kafka-2-1-1-1-2-1-1

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!