Defining Apache Avro Schema fullname in Apache NiFi

问题

Using NiFi 1.7.1 (which uses Java Avro 1.8.1) and in the AvroSchemaRegistry, I'm trying to define a schema which has the fields name and app.name at the top level. According to the Avro docs[1] I would assume that I could just define the fullname like normal "name": "app.name" but I hit the error Illegal character in: app.name. It's true that the name portion of the fullname does not allow dots but according to the docs: "If the name specified contains a dot, then it is assumed to be a fullname..."

I then tried using the namespace field. Using the following schema:

{
  "type": "record",
  "name": "nameRecord",
  "fields": [
    {
      "type": [
        "string",
        "null"
      ],
      "name": "name"
    },
    {
      "type": [
        "string",
        "null"
      ],
      "namespace": "app",
      "name": "name"
    }
  ]
}

I hit this error: Duplicate field name in record nameRecord: name type:UNION pos:1 and name type:UNION pos:0

Ultimately, I'd like to be able to define a schema for record like this (in JSON):

{
  "name": "Joe",
  "app.name": "NiFi"
}

[1] https://avro.apache.org/docs/1.8.1/spec.html#names

回答1:

According to the docs, namespaces are only supported for record, enum, and fixed types, and other fields must adhere to the "regular" naming conventions for which a period (.) is not a valid character.

However as of NiFi 1.5.0 (via NIFI-4612), you could specify the schema in an AvroSchemaRegistry, and set "Validate Field Names" to false. This should allow you to bypass the restriction of having a field's name be app.name.

来源：https://stackoverflow.com/questions/52340069/defining-apache-avro-schema-fullname-in-apache-nifi

标签

avro

apache-nifi