Apache Beam Counter/Metrics not available in Flink WebUI

Question

I\'m using Flink 1.4.1 and Beam 2.3.0, and would like to know is it possible to have metrics available in Flink WebUI (or anywhere at all), as in Dataflow WebUI ?

I\'ve

Answer 1

If your pipeline is running in detached mode, metrics are not supported to be queried. Refer this.

public class FlinkDetachedRunnerResult implements PipelineResult {

  FlinkDetachedRunnerResult() {}

  @Override
  public State getState() {
    return State.UNKNOWN;
  }

  @Override
  public MetricResults metrics() {
    throw new UnsupportedOperationException("The FlinkRunner does not currently support metrics.");
  }

  @Override
  public State cancel() throws IOException {
    throw new UnsupportedOperationException("Cancelling is not yet supported.");
  }

  @Override
  public State waitUntilFinish() {
    return State.UNKNOWN;
  }

  @Override
  public State waitUntilFinish(Duration duration) {
    return State.UNKNOWN;
  }

  @Override
  public String toString() {
    return "FlinkDetachedRunnerResult{}";
  }
}

However, I was able to view the metrics using slf4j reporter

Answer 2

Once you have selected a job in your dashboard, you will see the DAG for that job and below the DAG there are a list of tabs.

1. Click on "Task Metrics" Tab
2. Click on a box of your DAG
3. Click on the Add Metric Button, to show that operator metric