Apache Flink Dashboard not showing metrics

混江龙づ霸主 提交于 2021-02-07 21:48:29

问题


I have the following very simple Apache Flink Pipeline for which I would like to get some metrics, as explained in the Apache Flink documentation, via the Apache Flink Dashboard:

import org.apache.flink.api.common.functions.RichMapFunction;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.metrics.Counter;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.source.RichSourceFunction;

public class Pipeline {

    public static void main(String[] args) throws Exception {
        final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        env.addSource(new RichSourceFunction<String>() {
            private static final long serialVersionUID = 3990963645875188158L;
            private boolean notCanceled = true;

            @Override
            public void run(SourceContext<String> ctx) throws Exception {
                while (notCanceled) {
                    ctx.collect("test");
                }
            }

            @Override
            public void cancel() {
                notCanceled = false;
            }
        }).map(new RichMapFunction<String, String>() {
            private static final long serialVersionUID = 1L;
            private transient Counter counter;

            @Override
            public void open(Configuration parameters) throws Exception {
                super.open(parameters);
                this.counter = getRuntimeContext()
                        .getMetricGroup()
                        .counter("myCounter");
            }

            @Override
            public String map(String value) throws Exception {
                this.counter.inc();
                return "mappedtext";
            }
        }).print();

        env.execute();
    }

}

I do run that Pipeline using the Docker Setup available via Docker-Hub. Everything uses Apache Flink 1.10.0. The Pipeline runs fine, but when I try to view my metric I only get:

Now I ask myself what am I doing wrong? Is there some configuration parameter I am missing or is this not the correct place to view that metric in the dashboard? Can someone please advise or at least point me to a resource, where I would get more information?


回答1:


I believe what's going on is that you have added an operator metric to the map operator, but the web ui is displaying task metrics. (In the case of this simple, embarrassingly parallel job, the source, map, and sink operators have been chained together into a single task.)

To inspect this metric you've added, you could use the REST API, or any of the metrics reporters. I think it may also show up in the web UI if you disable operator chaining via

    env.disableOperatorChaining();


来源:https://stackoverflow.com/questions/60490957/apache-flink-dashboard-not-showing-metrics

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!