What does the shuffling phase actually do?

Question

What does the shuffling phase actually do?

---

Possibility - A

As shuffling is the process of bringing the mapper o/p to the reducer o/p, it just brings t

Answer 1

Mappers and Reducers are not separate machines but just separate code. Both, the mapping code as well as the reducing code runs on the same set machines present in the cluster.

---

So, after all machines in the cluster have run mapper, the results are:

1. Binned locally on the node (Consider it a "local-grouping"); and,
2. Shuffled/Redistributed across all nodes on the cluster.

Consider the step-2 a "global-grouping" because it is done in a manner that all values belonging to one key, go to their assigned unique node.

Now, the nodes run the Reducer code on the (key, value) pairs residing on their memory.