Recursive schema with avro (SchemaBuilder)

孤人 提交于 2019-12-25 03:53:40

问题


Is it possible to make an avro schema which is recursive, like

Schema schema = SchemaBuilder
    .record("RecursiveItem")
    .namespace("com.example")
    .fields()
    .name("subItem")
    .type("RecursiveItem")
    .withDefault(null) // not sure about that too...
    .endRecord();

I get a StackOverflowError when using it like that:

static class RecursiveItem {
  RecursiveItem subItem;
}

RecursiveItem item1 = new RecursiveItem();
RecursiveItem item2 = new RecursiveItem();
item1.subItem = item2;

final DatumWriter<RecursiveItem> writer = new SpecificDatumWriter<>(schema);

// note: I actually want a binary output, but I started with some json code I found
ByteArrayOutputStream stream = new ByteArrayOutputStream();
final JsonEncoder encoder = EncoderFactory.get().jsonEncoder(schema, stream);
writer.write(rec1, encoder);
String json = stream.toString();

Note: I also get StackOverflowError if I make the schema using:

Schema schema = ReflectData.get().getSchema(RecursiveItem.class);

回答1:


Warning: I found a solution to write, but cannot read it bask :-\

I am not sure to really understand, but I manage to make it works with:

  1. ReflectDatumWriter should be used instead of SpecificDatumWriter

  2. I still had issue with schema not found due to automated lookup of schema upon encoding. It looks for a schema with a namespace+name derived automatically. And in my case where the class is a static subclass, the following should be used:

    String cls = RecursiveItem.class.getSimpleName();
    String pck = RecursiveItem.class.getPackage().getName();
    if (RecursiveItem.class.getEnclosingClass() != null) // nested class
      pck = RecursiveItem.class.getEnclosingClass().getName() + "$";
    
  3. To manage the null, the following schema should be used

    Schema schema0 = SchemaBuilder
      .record(cls)
      .namespace(pck)
      .fields()
      .name("subItem")
      .type().unionOf().nullType().and().type("RecursiveItem").endUnion()
      .nullDefault()
      .endRecord();
    


来源:https://stackoverflow.com/questions/56681506/recursive-schema-with-avro-schemabuilder

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!