Flink序列化器

↘锁芯ラ 提交于 2020-08-18 23:20:05

如果应用使用的google protobuf 或 apache thrift序列器工具, 你是需要注册自已的序列化工具的。以protobuf和thrift为例,示例如下:
譬如 google protobuf 样例:

  1. 注册ProtobufSerializer序列化器:
    final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); env.getConfig().registerTypeWithKryoSerializer(PbSdkStat.DataRecords.class, ProtobufSerializer.class);
  2. 添加maven依赖

<dependency>
  <groupId>com.twitter</groupId>
  <artifactId>chill-protobuf</artifactId>
  <version>0.7.6</version>
  <!-- exclusions for dependency conversion -->
  <exclusions>
  <exclusion>
   <groupId>com.esotericsoftware.kryo</groupId>
   <artifactId>kryo</artifactId>
  </exclusion>
  </exclusions>
</dependency>
<!-- We need protobuf for chill-protobuf -->
<dependency>
  <groupId>com.google.protobuf</groupId>
  <artifactId>protobuf-java</artifactId>
  <version>3.7.0</version>
</dependency>

譬如apache Thrift样例

  1. 注册TBaseSerializer序列化器:
    ``final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
    env.getConfig().addDefaultKryoSerializer(MyCustomType.class, TBaseSerializer.class);
  2. 添加maven依赖

<dependency>
 <groupId>com.twitter</groupId>
 <artifactId>chill-thrift</artifactId>
 <version>0.7.6</version>
 <!-- exclusions for dependency conversion -->
 <exclusions>
  <exclusion>
   <groupId>com.esotericsoftware.kryo</groupId>
   <artifactId>kryo</artifactId>
  </exclusion>
 </exclusions>
</dependency>
<!-- libthrift is required by chill-thrift -->
<dependency>
 <groupId>org.apache.thrift</groupId>
 <artifactId>libthrift</artifactId>
 <version>0.11.0</version>
 <exclusions>
  <exclusion>
   <groupId>javax.servlet</groupId>
   <artifactId>servlet-api</artifactId>
  </exclusion>
  <exclusion>
   <groupId>org.apache.httpcomponents</groupId>
   <artifactId>httpclient</artifactId>
  </exclusion>
 </exclusions>
</dependency>

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!