avro4s

Creating AVRO schema from JSON Schema File

由 可紊 提交于 2020-05-27 03:18:36
问题 I have the JSON file & JSON Schema to be parsed into the AVRO Schema. I am little bit confused, do i have to write the manual AVRO schema using the data types defined in AVRO documentation. Or is there any automated method / function / program that can work exactly the same as required ? 回答1: Well, you can try https://github.com/fge/json-schema-avro, but they say it is still not complete. So not sure it will work, though 回答2: avro4s derives schemas at compile time from case classes: import

Creating AVRO schema from JSON Schema File

由 我的未来我决定 提交于 2020-05-27 03:17:28
问题 I have the JSON file & JSON Schema to be parsed into the AVRO Schema. I am little bit confused, do i have to write the manual AVRO schema using the data types defined in AVRO documentation. Or is there any automated method / function / program that can work exactly the same as required ? 回答1: Well, you can try https://github.com/fge/json-schema-avro, but they say it is still not complete. So not sure it will work, though 回答2: avro4s derives schemas at compile time from case classes: import

Data validation in AVRO

由 痞子三分冷 提交于 2020-01-11 05:26:07
问题 I am new to AVRO and please excuse me if it is a simple question. I have a use case where I am using AVRO schema for record calls. Let's say I have avro schema { "name": "abc", "namepsace": "xyz", "type": "record", "fields": [ {"name": "CustId", "type":"string"}, {"name": "SessionId", "type":"string"}, ] } Now if the input is like { "CustId" : "abc1234" "sessionID" : "000-0000-00000" } I want to use some regex validations for these fields and I want take this input only if it comes in

Avro serialization with generic type issue

由 余生颓废 提交于 2019-12-11 07:58:01
问题 I need to write a function in Scala that returns an Array of byte serializated with AvroOutputStream, but in scala i can't get the class of the generic object i'm passing in input. Here is my util class: class AvroUtils { def createByteArray[T](obj: T): Array[Byte] = { val byteArrayStream = new ByteArrayOutputStream() val output = AvroOutputStream.binary[T](byteArrayStream) output.write(obj) output.close() byteArrayStream.toByteArray() } } As you can see if tou test this code is that

Data validation in AVRO

由 僤鯓⒐⒋嵵緔 提交于 2019-12-01 07:33:56
I am new to AVRO and please excuse me if it is a simple question. I have a use case where I am using AVRO schema for record calls. Let's say I have avro schema { "name": "abc", "namepsace": "xyz", "type": "record", "fields": [ {"name": "CustId", "type":"string"}, {"name": "SessionId", "type":"string"}, ] } Now if the input is like { "CustId" : "abc1234" "sessionID" : "000-0000-00000" } I want to use some regex validations for these fields and I want take this input only if it comes in particular format shown as above. Is there any way to specify in avro schema to include regex expression? Any
订阅 avro4s