What runs first: the partitioner or the combiner?

后端 未结 8 569
星月不相逢
星月不相逢 2020-12-29 13:21

I was wondering between partitioner and combiner, which runs first?

I was of the opinion it is the partitiner first and then combiner and then the keys are redirecte

相关标签:
8条回答
  • 2020-12-29 14:21

    The direct answer to your question is => COMBINER

    Details: Combiner can be viewed as mini-reducers in the map phase. They perform a local-reduce on the mapper results before they are distributed further. Once the Combiner functionality is executed, it is then passed on to the Reducer for further work.

    where as

    The partitioner comes into the picture when we are working one more than on reducer. So, the partitioner decides which reducer is responsible for a particular key. They basically take the Mapper Result(if Combiner is used then Combiner Result) and send it to the responsible Reducer based on the key.

    For a better understanding you can refer the following image, which I have taken from Yahoo Developer Tutorial on Hadoop.
    (source: flickr.com)

    Here is the tutorial .

    0 讨论(0)
  • 2020-12-29 14:24

    Mapper -> Combiner -> Partitionar -> Reducer

    0 讨论(0)
提交回复
热议问题