Avro with Java 8 dates as logical type

Question

Latest Avro compiler (1.8.2) generates java sources for dates logical types with Joda-Time based implementations. How can I configure Avro compiler to produce sources that u

Answer 1

AVRO 1.10 added support for LocalDateTime, see Apache Avro™ 1.10.0 Specification

Answer 2

Currently (avro 1.8.2) this is not possible. It's hardcoded to generate Joda date/time classes.

The current master branch has switched to Java 8 and there is an open issue (with Pull Request) to add the ability to generate classes with java.time.* types.

I have no idea on any kind of release schedule for whatever is currently in master unfortunately. If you feel adventurous you can apply the patch to 1.8.2, since in theory it should all be compatible. The underlying base types when serializing / deserializing are still integers and longs.

Answer 3

You need to create your own Conversions to support java-8 date-time api, below is a conversion for java.time.LocalDate:

class Java8LocalDateConversion extends Conversion<LocalDate> {
    @Override
    public Class<LocalDate> getConvertedType() {
        return LocalDate.class;
    }

    @Override
    public String getLogicalTypeName() {
        //      v--- reuse the logical type `date`
        return "date";
    }

    @Override
    // convert LocalDate to Integer
    public Integer toInt(LocalDate value, Schema schema, LogicalType type) {
        return (int) value.toEpochDay();
    }

    @Override
    // parse LocalDate from Integer
    public LocalDate fromInt(Integer value, Schema schema, LogicalType type) {
        return LocalDate.ofEpochDay(value);
    }
}

The logical types can be reused in avro, so you can using the existing date logical type, for example:

Schema schema = LogicalTypes.date().addToSchema(Schema.create(Type.INT));

For the serializing & deserializing you should set the GenericData which will find your own conversion, for example:

//serializing
DatumWriter<T> out = new SpecificDatumWriter<>(schema, data());

// deserializing
DatumReader<T> in = new SpecificDatumReader<>(schema, schema, data());

private SpecificData data() {
    SpecificData it = new SpecificData();
    it.addLogicalTypeConversion(new Java8LocalDateConversion());
    return it;
}

If you don't want to configure the GenericData every time, you can use the global GenericData instead, for example:

//      register the conversion globally ---v
SpecificData.get().addLogicalTypeConversion(new Java8LocalDateConversion());

Answer 4

With avro 1.9.2 you can use e.g. date for LocalDate:

    {
      "name": "Transaction",
      "type": "record",
      "fields": [
        {
          "name": "time",
          "type": {
            "type": "int",
            "logicalType": "date"
          }
        },

See Logical Types for the other types.