问题
As we know that Hadoop
guarantees that the same keys which come from different mappers
will be sent to the same reducer
.
But if two different keys have the same hash value, they definitely will go to the same reducer
, so will them be sent to the same reduce
method orderly? Which part is responsible for this logic?
Thanks a lot!
回答1:
Q1:so will them be sent to the same reduce method orderly
Ans : yes
Q2:Which part is responsible for this logic
Ans : shuffle sort
Example :
key value
1 2
1 2
2 5
3 19
6 20
Lets say number of reducer is 5 so now .
Reduce 0 will get key NO key-value pairs
Reduce 1 will get key 1,6 in same order
Reduce 2 will get key 2
Reduce 3 will get key 3
Reduce 4 will get key NO key-value pairs
来源:https://stackoverflow.com/questions/20757318/what-two-different-keys-go-to-the-same-reducer-by-the-default-hash-partitioner-i