apache-flink

问题 I have the following very simple Apache Flink Pipeline for which I would like to get some metrics, as explained in the Apache Flink documentation, via the Apache Flink Dashboard: import org.apache.flink.api.common.functions.RichMapFunction; import org.apache.flink.configuration.Configuration; import org.apache.flink.metrics.Counter; import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; import org.apache.flink.streaming.api.functions.source.RichSourceFunction; public

问题 I have the following very simple Apache Flink Pipeline for which I would like to get some metrics, as explained in the Apache Flink documentation, via the Apache Flink Dashboard: import org.apache.flink.api.common.functions.RichMapFunction; import org.apache.flink.configuration.Configuration; import org.apache.flink.metrics.Counter; import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment; import org.apache.flink.streaming.api.functions.source.RichSourceFunction; public

问题 For example A should be followed by B within 10 seconds. I know how to track if this DID occur (.next, .within), but I want to send an alert if B never happened within the window. public static void main(String[] args) throws Exception { final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); // checkpointing is required for exactly-once or at-least-once guarantees // env.enableCheckpointing(1000); final RMQConnectionConfig connectionConfig = new

问题 I am just trying to understand the use case when to use CoProcessFunction in Flink. Explanation with an example would help me to understand the concept better. 回答1: A CoProcessFunction is similar to a RichCoFlatMap, but with the addition of also being able to use timers. The timers are useful for expiring state for stale keys, or for raising alarms when keep alive messages fail to arrive, for example. A CoProcessFunction allows you to use one stream to influence how another is processed, or

问题 Is there anyway to use a python user defined function within a Java Flink Job or anyway to communicate for example the result of a transformation done by flink with java with a python user defined function to apply some machine learning things: I know that from pyFlink you can do something like this: table_env.register_java_function("hash_code", "my.java.function.HashCode") But I need to do something like that but add the python function from java, or how can I pass the result of a java

问题 I used the following code to connect Flink to ElasticSearch. But when running with Flink, a lot of errors are displayed.The program first enters the data from a port and then reads each line in the command line according to the program written. It then displays the number of words. The main problem is when connecting to a elasticsearch that unfortunately gives error when connecting. Are these errors? What classes do you need to connect Minimal Flink to Elastic Search? public class Elastic {

问题 I used the following code to connect Flink to ElasticSearch. But when running with Flink, a lot of errors are displayed.The program first enters the data from a port and then reads each line in the command line according to the program written. It then displays the number of words. The main problem is when connecting to a elasticsearch that unfortunately gives error when connecting. Are these errors? What classes do you need to connect Minimal Flink to Elastic Search? public class Elastic {

问题 I used the following code to connect Flink to ElasticSearch. But when running with Flink, a lot of errors are displayed.The program first enters the data from a port and then reads each line in the command line according to the program written. It then displays the number of words. The main problem is when connecting to a elasticsearch that unfortunately gives error when connecting. Are these errors? What classes do you need to connect Minimal Flink to Elastic Search? public class Elastic {

问题 I have a flink job, with parallelism set to 6, few simple transformations and the issue is that when Flink is been running for more than 12 hours for example the Load on the machine start to increase, then I thought that was because of the traffic into flink during some hours of the day, but the issue is that when the traffic goes down, the load on the machine continue a bit higher, lower than before but still higher. Use cases: DataStream<Event> from_source = rabbitConsumer .flatMap(new

问题 i have to fetch data from Rest API from flink process element in every data fetch from the stream, how can i achive that, i couldnt find enough meterials to call the Rest service asynchronously. please help me with some sample articles. 回答1: All the job is happening inside the asyncInvoke of the RichAsyncFunction . So, to be able to call REST service, You need to use some async HTTP client (technically it could be a synchronous client but this doesn't make sense). An example of async http