Kafka - error when producing from command line (character ('<' (code 60)): expected a valid value)

Question

I spinned on my laptop a Kafka in Docker (with docker-compose).

After that, created new kafka topic with:

kafka-topics --zookeeper localhost:2181  --create --topic simple    --replication-factor 1 --partitions 1

(did not create schema in Schema Registry yet).

Now trying to produce (based on this example - step 3 - https://docs.confluent.io/4.0.0/quickstart.html):

kafka-avro-console-producer \
         --broker-list localhost:9092 --topic simple \
         --property value.schema='{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}'

Entering value:

{"f1": "value1"}

Error:

{"f1": "value1"}
org.apache.kafka.common.errors.SerializationException: Error registering Avro schema: {"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}
Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
 at [Source: (sun.net.www.protocol.http.HttpURLConnection$HttpInputStream); line: 1, column: 2]; error code: 50005
    at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:191)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:218)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:307)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:299)
    at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:294)
    at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:61)
    at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:100)
    at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:79)
    at io.confluent.kafka.formatter.AvroMessageReader.readMessage(AvroMessageReader.java:166)
    at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:59)
    at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)

How to solve this?
Can it be because Kafka cluster uses SSL but error is bogus? Thanks.

Answer 1

kafka-avro-console-producer, by default, assumes that Schema Registry is listening at port http://localhost:8081. However, if another process is listening to that port, weird errors might occur.

In order to overcome this issue, you can specify the schema URL when running the producer using --property schema.registry.url=http://localhost:18081

For example,

kafka-avro-console-producer \
         --broker-list localhost:9092 --topic simple \
         --property value.schema='{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}' \
         --property schema.registry.url=http://localhost:18081