debezium

问题 I'd like to read from a multiple topics from cdc debezium from source postgres database, using a key from kafka message holding a primary keys. Then, the connector performs ETL operations in source database. When I set delete.enabled to true I cannot use kafka primary keys, it says I have to specify record_key and pk_fields . My idea is, set regex to read multiple desired topics, get table name from topic name and use primary keys holding by kafka topic, which is being currently read. name

问题 We are currently trying to use Debezium for capturing changes of 4 tables in a Postgres database. We are currently aware that for this use case we could use a kafka-streams app to join/aggregate tables for a KTable, however we want to keep kafka-stream topology simple, so the idea would be to use a Materialized View from Postgres and capture it changes. Is it possible to do this, if so than how should we configure it? Kafka Connect Source Configuration: { "name": "campaign-db-source-connector

问题 What are the differences between JDBC Connector and Debezium SQL Server CDC Connector (or any other relational database connector) and when should I choose one over another, searching for a solution to sync between two relational databases? Not sure if this discussion should be about CDC vs JDBC Connector, and not Debezium SQL Server CDC Connector, or even just Debezium, looking forward for later editing, depends on the given answers (Though my case is about SQL Server sink). Sharing with you

问题 I'm trying to connect my SQL Server database to kafka on windows, i downloaded Debezium jar files, they are in a folder named debezium-connector-sqlserver, here's what it looks like : Kafka folder, Debezium folder I also added this line plugin.path=C:\\current_kafka_2.12-2.0.0\\debezium-connector-sqlserver in connect-standalone.properties, it's where i put all the jar files i downloaded from Debezium. i also created a file named connect-mssql.properties, and i put this in it. name=inventory

问题 Using postgres source connector in kafka. It works properly for some time and suddently stops with above error. Please assist if someone knows this issue. 回答1: This happens if the database is not available and the only way to fix it to restart the connector. I would advise you to check the database logs and see if database is going down or refusing connection to Kafka Connect 来源： https://stackoverflow.com/questions/60450498/org-apache-kafka-connect-errors-connectexception-an-exception

问题 Are there any best practices that are followed for whitelisting a new table to the debezium mysql connector ? We are using debezium mysql connector for our CDC flows and a use case has arised to whitelist more tables to the connector configuration. Here are the version details of the debezium being used and the { "class": "io.debezium.connector.mysql.MySqlConnector", "version": "0.8.0.Final", "snapshot.mode": "schema_only" } There is a debezium ticket https://issues.redhat.com/browse/DBZ-906

问题 I use debezium cdc connect pg, and i build the pg 11 use by docker，the pg is run well. when i use debezium in kafka connector， it report： Couldn't obtain encoding for database test the curl is: curl -H "Accept: application/json" -H "Content-type: application/json" -X POST http://localhost:8083/connectors/ -d '{ "name": "debezium", "config": { "name": "debezium", "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "tasks.max": "1", "database.hostname": "localhost",

问题 I use debezium cdc connect pg, and i build the pg 11 use by docker，the pg is run well. when i use debezium in kafka connector， it report： Couldn't obtain encoding for database test the curl is: curl -H "Accept: application/json" -H "Content-type: application/json" -X POST http://localhost:8083/connectors/ -d '{ "name": "debezium", "config": { "name": "debezium", "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "tasks.max": "1", "database.hostname": "localhost",

问题 I use debezium cdc connect pg, and i build the pg 11 use by docker，the pg is run well. when i use debezium in kafka connector， it report： Couldn't obtain encoding for database test the curl is: curl -H "Accept: application/json" -H "Content-type: application/json" -X POST http://localhost:8083/connectors/ -d '{ "name": "debezium", "config": { "name": "debezium", "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "tasks.max": "1", "database.hostname": "localhost",

问题 When I'm trying to connect Debezium to my SQL Server database after enable CDC feature, I have this error message : java.lang.RuntimeException: Couldn't obtain database name, at io.debezium.connector.sqlserver.SqlServerConnection.retrieveRealDatabaseName(SqlServerConnection.java:364), at io.debezium.connector.sqlserver.SqlServerConnection.<init>(SqlServerConnection.java:84), at io.debezium.connector.sqlserver.SqlServerConnectorTask.start(SqlServerConnectorTask.java:86), at io.debezium