confluent-platform

问题 Occasionally with Kafka Connect, I see my JdbcSourceConnector's task go up and down--that is, the REST interface sometimes reports one task that is RUNNING and sometimes reports no tasks (the connector remains RUNNING this whole time). During these times, the task seems to be working when its running. Then, if I delete and re-create the connector, the problem seems to go away. I suspect something is wrong--that tasks shouldn't churn like this, right? But the INFO/WARN logs on the server don't

问题 Occasionally with Kafka Connect, I see my JdbcSourceConnector's task go up and down--that is, the REST interface sometimes reports one task that is RUNNING and sometimes reports no tasks (the connector remains RUNNING this whole time). During these times, the task seems to be working when its running. Then, if I delete and re-create the connector, the problem seems to go away. I suspect something is wrong--that tasks shouldn't churn like this, right? But the INFO/WARN logs on the server don't

问题 I've been using KSQL and so far it's been working great. But now I'd like to sink the output to BigQuery via Kafka Connect, and need to attach a JSON schema. I'm having trouble figuring out how to do this. Here's my query: CREATE STREAM tweets_original ( CreatedAt BIGINT, Id BIGINT, Text VARCHAR, Source VARCHAR, GeoLocation VARCHAR, User STRUCT<Id BIGINT, Name VARCHAR, Description VARCHAR, ScreenName VARCHAR, URL VARCHAR, FollowersCount BIGINT, FriendsCount BIGINT> ) WITH (kafka_topic='tweets

问题 Status My HDFS was installed via ambari, HDP. I'm Currently trying to load kafka topics into HDFS sink. Kafka and HDFS was installed in the same machine x.x.x.x. I didn't change much stuff from the default settings, except some port that according to my needs. Here is how i execute kafka: /usr/hdp/3.1.4.0-315/kafka/bin/connect-standalone.sh /etc/kafka/connect-standalone.properties /etc/kafka-connect-hdfs/quickstart-hdfs.properties Inside connect-standalone.properties bootstrap.servers=x.x.x.x

问题 Status My HDFS was installed via ambari, HDP. I'm Currently trying to load kafka topics into HDFS sink. Kafka and HDFS was installed in the same machine x.x.x.x. I didn't change much stuff from the default settings, except some port that according to my needs. Here is how i execute kafka: /usr/hdp/3.1.4.0-315/kafka/bin/connect-standalone.sh /etc/kafka/connect-standalone.properties /etc/kafka-connect-hdfs/quickstart-hdfs.properties Inside connect-standalone.properties bootstrap.servers=x.x.x.x

问题 I am using following program to produce records in kafka: import java.io.IOException; import java.security.SecureRandom; public class SensorStatusProducer { private final static String TOPIC = "SENSOR_STATUS_DETAILS"; private final static String PRODUCER_URI = "http://xxx.xxx.xxx.xxx:8082/topics/" + TOPIC; private final static SecureRandom randomNumber = new SecureRandom(); private final static SensorDetails sensorDetails = new SensorDetails(); public static void main(String[] args) { int[]

问题 I have produced following data on topic named SENSOR_STATUS_DETAILS in json : 1001 { "sensorid": 1001, "status": "CONNECTED", "lastconnectedtime": "2020-05-31 22:31:54" } 1002 { "sensorid": 1002, "status": "CONNECTED", "lastconnectedtime": "2020-05-31 22:33:37" } I am trying to make a table from it as: CREATE TABLE STATUS_IB_TABLE (ROWKEY INT KEY, sensorid INTEGER, status VARCHAR, lastconnectedtime STRING) WITH (TIMESTAMP='lastconnectedtime', TIMESTAMP_FORMAT='yyyy-MM-dd HH:mm:ss', KAFKA

问题 I have produced following data on topic named SENSOR_STATUS_DETAILS in json : 1001 { "sensorid": 1001, "status": "CONNECTED", "lastconnectedtime": "2020-05-31 22:31:54" } 1002 { "sensorid": 1002, "status": "CONNECTED", "lastconnectedtime": "2020-05-31 22:33:37" } I am trying to make a table from it as: CREATE TABLE STATUS_IB_TABLE (ROWKEY INT KEY, sensorid INTEGER, status VARCHAR, lastconnectedtime STRING) WITH (TIMESTAMP='lastconnectedtime', TIMESTAMP_FORMAT='yyyy-MM-dd HH:mm:ss', KAFKA

问题 I have a stream deriving from a topic that contains 271 total messages the stream also contains 271 total messages, but when i create a other stream from that previous stream to flatten it, i get total messages of 542=(271*2). this is the stream deriving from the topic Name : TRANSACTIONSPURE Type : STREAM Key field : Key format : STRING Timestamp field : Not set - using <ROWTIME> Value format : JSON Kafka topic : mongo_conn.digi.transactions (partitions: 1, replication: 1) Field | Type

问题 Stream : describe ammas; Field | Type ------------------------------------- ROWTIME | BIGINT (system) ROWKEY | VARCHAR(STRING) (system) ID | INTEGER ------------------------------------- For runtime statistics and query details run: DESCRIBE EXTENDED <Stream,Table>; Table-01 : ksql> show tables; Table Name | Kafka Topic | Format | Windowed ------------------------------------------------- ANNAT | anna | DELIMITED | false APPAT | appa | DELIMITED | false ---------------------------------------