WindowFunction cannot be applied using WindowStream.apply() function

不打扰是莪最后的温柔 提交于 2019-12-24 00:54:45

问题


I'm relatively new to using Apache Flink and Scala, and am just getting to grips with some of the basic functionality. I've hit a wall trying to implement a custom WindowFunction.

The problem is that when I try to implement a custom WindowFunction the IDE gives an error on the ".apply()" function

Cannot resolve symbol apply

Unspecified value parameters: foldFunction: (NotInferedR, Data.Fingerprint) => NotInferedR, windowFunction: (Tuple, TimeWindow, Iterable[NotInferedR], Collector[NotInferedR]) => Unit 
Unspecified value parameters: foldFunction: FoldFunction[Data.Fingerprint, NotInferedR], function: WindowFunction[NotInferedR, NotInferedR, Tuple, TimeWindow] 
Unspecified value parameters: function: WindowFunction[Data.Fingerprint, NotInferedR, Tuple, TimeWindow] 
Unspecified value parameters: windowFunction: (Tuple, TimeWindow, Iterable[Data.Fingerprint], Collector[NotInferedR]) => Unit 

Type mismatch, expected: (Tuple, TimeWindow, Iterable[Data.Fingerprint], Collector[NotInferedR]) => Unit, actual: DataTimeWindow.DataWindow 
Type mismatch, expected: WindowFunction[Data.Fingerprint, NotInferedR, Tuple, TimeWindow], actual: DataTimeWindow.DataWindow

This is my code:

val test = hashMap
      .keyBy("hash")
      .timeWindow(Time.minutes(1))
      .apply(new DataWindow())

And this is the WindowFunction:

class DataWindow extends WindowFunction[Data.Fingerprint, String, String, TimeWindow]  {

    override def apply(key: String,
                       window: TimeWindow,
                       input: Iterable[Fingerprint],
                       out: Collector[String]) {

      out.collect("helo")
    }
  }

回答1:


I think the problem is with the third type parameter of the WindowFunction, i.e., the type of the key. Because you declare the key in the keyBy method as String (keyBy("hash")), it is not possible to determine the type of the key at compile time. There are two options to resolve this:

  1. Use a KeySelector function in keyBy to extract the key (something like keyBy(x: FingerPrint => x.hash)). The return type of the KeySelector function is known at compile time such that you can use a typed WindowFunction.
  2. Change the type of the third type parameter of the WindowFunction to Tuple. Tuple is a generic holder for the keys extracted by keyBy. In your case it will be a Tuple1 and the hash String can be accessed by Tuple1.f0.


来源:https://stackoverflow.com/questions/47033981/windowfunction-cannot-be-applied-using-windowstream-apply-function

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!