MapReduce sort by value in descending order

Question

I\'m trying to write in pseudo code a MapReduce task that returns the items sorted in descending order. For example: for the wordcount task, instead of getting:

Answer 1

Here you can take help of below reducer code to achieve sorting in descending order .

Assuming you have written mapper and driver code where mapper will produce output as (Banana,1) etc

In reducer we will sum all values for a particular key and put final result in a map then sort the map on the basis of values and write final result in cleanup function of reduce.

Please see below code for further understadnind:

public class Word_Reducer extends Reducer {
    // Change access modifier as per your need 
    public Map map = new LinkedHashMap();
    public void reduce(Text key , Iterable values ,Context context)
    { 
    // write logic for your reducer 
    // Enter reduced values in map for each key
    for (IntWritable value : values ){
         // calculate "count" associated with each word 
    }
    map.put(key.toString() , count); 
}

public void cleanup(Context context){ 
    //Cleanup is called once at the end to finish off anything for reducer
    //Here we will write our final output
    Map  sortedMap = new HashMap();    
    sortedMap = sortMap(map);

    for (Map.Entry entry = sortedMap.entrySet()){
        context.write(new Text(entry.getKey()),new IntWritable(entry.getValue()));
    }
}

public Map sortMap (Map unsortMap){
    Map hashmap = new LinkedHashMap();
    int count=0;
    List> list = new 
    LinkedList>(unsortMap.entrySet());
    //Sorting the list we created from unsorted Map
    Collections.sort(list , new Comparator>(){
        public int compare (Map.Entry o1 , Map.Entry o2 ){
            //sorting in descending order
            return o2.getValue().compareTo(o1.getValue());
        }
    });

    for(Map.Entry entry : list){
        // only writing top 3 in the sorted map 
        if(count>2)
            break;
        hashmap.put(entry.getKey(),entry.getValue());
    }
    return hashmap ; 
}