Flink webui when running from IDE

问题

I am trying to see my job in the web ui.

I use createLocalEnvironmentWithWebUI, code is running well in IDE, but impossible to see my job in http://localhost:8081/#/overview

  val conf: Configuration = new Configuration()
  import org.apache.flink.configuration.ConfigConstants
  conf.setBoolean(ConfigConstants.LOCAL_START_WEBSERVER, true)
  val env =  StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(conf)
  env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)


  val rides = env.addSource(
    new TaxiRideSource("nycTaxiRides.gz", 1,100))//60, 600))

  val filteredRides = rides
    .filter(r => GeoUtils.isInNYC(r.startLon, r.startLat) && GeoUtils.isInNYC(r.endLon, r.endLat))
    .map(r => (r.passengerCnt, 1))
    .keyBy(_._1)
    .window(TumblingTimeWindows.of(Time.seconds(5)))
    .sum(1)
    .map(r => (r._1.toString+"test", r._2))

  filteredRides.print()
  env.execute("Taxi Ride Cleansing")

Did I need to setup something else?

回答1:

I was able to start the Flink webui from IntelliJ by adding flink-runtime-web to the dependencies for my project. I did this by adding this to my pom.xml file:

<dependency>
    <groupId>org.apache.flink</groupId>
    <artifactId>flink-runtime-web_2.11</artifactId>
    <version>${flink.version}</version>
</dependency>

You also then need to create a local execution environment that includes the WebUI:

Configuration conf = new Configuration();
env = StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(conf);

回答2:

As of Flink version 1.5.0 adding the dependency mentioned before and using the following piece of code to start the StreamEnvironment worked for me:

Configuration config = new Configuration();
config.setBoolean(ConfigConstants.LOCAL_START_WEBSERVER, true);
StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(config);

While the processing is running, the web UI is available under http://localhost:8081

回答3:

Yes, if you want to use WebUI Dashboard, then you need to create an executable jar and then submit this jar to Flink dashboard. I will explain you this step by step

Step 1: Creating the jar from IDE code

you may need to change your execution environment to

StreamExecutionEnvironment envrionment = StreamExecutionEnvironment.getExecutionEnvironment();

In case you have multiple jars, then set the main class in Main-Class: variable of Manifest.mf file
Then create a jar using build artifacts in your IDE

Step 2: Start flink-local cluster which will show you dashboard.

I will assume that you have not downloaded the Flink binary, you can easily download it here, if you have Macintosh, I will suggest you to use brew install apache-flink which will download the latest stable release which is 1.3.2 currently
Ok, now you have to go to path where flink is installed and start local cluster

Step# 3 : submitting the job

submit jar via submit new job option and then run it

来源：https://stackoverflow.com/questions/46988499/flink-webui-when-running-from-ide

标签

apache-flink

flink-streaming