Apache flink - Mini cluster - Windowing operator execution problem

倖福魔咒の 提交于 2019-12-12 01:25:28

问题


This turned out to be the problem for below question

Apache flink - job simple windowing problem - java.lang.RuntimeException: segment has been freed - Mini Cluster problem

So I wanted to ask by giving specific detail.

Adding a very simple windowing operator to job causes below error in MINI CLUSTER ENVIRONMENT:

Caused by: java.lang.RuntimeException: segment has been freed
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.emitWatermark(RecordWriterOutput.java:123)
    at org.apache.flink.streaming.runtime.tasks.OperatorChain$BroadcastingOutputCollector.emitWatermark(OperatorChain.java:603)
    at org.apache.flink.streaming.api.operators.AbstractStreamOperator$CountingOutput.emitWatermark(AbstractStreamOperator.java:691)
    at org.apache.flink.streaming.api.operators.AbstractStreamOperator.processWatermark(AbstractStreamOperator.java:759)
    at org.apache.flink.streaming.runtime.tasks.OperatorChain$ChainingOutput.emitWatermark(OperatorChain.java:479)
    ... 10 more
Caused by: java.lang.IllegalStateException: segment has been freed
    at org.apache.flink.core.memory.HybridMemorySegment.put(HybridMemorySegment.java:228)
    at org.apache.flink.core.memory.HybridMemorySegment.put(HybridMemorySegment.java:381)
    at org.apache.flink.runtime.io.network.buffer.BufferBuilder.append(BufferBuilder.java:85)
    at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.addRecord(SpanningRecordSerializer.java:97)
    at org.apache.flink.runtime.io.network.api.writer.RecordWriter.sendToTarget(RecordWriter.java:131)
    at org.apache.flink.runtime.io.network.api.writer.RecordWriter.broadcastEmit(RecordWriter.java:117)
    at org.apache.flink.streaming.runtime.io.StreamRecordWriter.broadcastEmit(StreamRecordWriter.java:87)
    at org.apache.flink.streaming.runtime.io.RecordWriterOutput.emitWatermark(RecordWriterOutput.java:121)

When I deploy it to test cluster this problem does NOT occur?

Is there any special setting for local execution?

Please advise

来源:https://stackoverflow.com/questions/54291125/apache-flink-mini-cluster-windowing-operator-execution-problem

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!