Kafka Connect with a JdbcConnectionSource connector fails to create task (connector is RUNNING but task is not)

Question

It seems like rather often I create a Kafka Connect connector from the JdbcConnectionSource based on a query, and the connector is created successsfully with status \"RUNNIN

Answer 1

How can one get more information to troubleshoot when a connector fails to create a RUNNING task?

I would increase the logging level on your Kafka Connect worker. Since you're using Apache Kafka 2.4 you can do this dynamically, which is rather useful. Issue this REST API call to your Kafka Connect worker:

curl -X PUT http://localhost:8083/admin/loggers/io.confluent \
     -H "Content-Type:application/json" -d '{"level": "TRACE"}'

This bumps up all messages for any Confluent connector to TRACE. It also returns a list of the individually loggers, from which you can cherry-pick different loggers and turn their specific loglevel up or down as required. For example:

curl -X PUT http://localhost:8083/admin/loggers/io.confluent.connect.jdbc.dialect.DatabaseDialects \
     -H "Content-Type:application/json" -d '{"level": "INFO"}'

Ref: https://rmoff.net/2020/01/16/changing-the-logging-level-for-kafka-connect-dynamically/