Working of RecordReader in Hadoop

Question

Can anyone explain how the RecordReader actually works? How are the methods nextkeyvalue(), getCurrentkey() and getprogress() work after t

Answer 1

org.apache.hadoop.mapred.MapTask - runNewMapper()

Imp steps:

1. creates new mapper

2. get input split for the mapper

3. get recordreader for the split

4. initialize record reader

5. using record reader iterate through getNextKeyVal() and pass key,val to mappers map method

6. clean up

Answer 2

(new API): The default Mapper class has a run method which looks like this:

public void run(Context context) throws IOException, InterruptedException {
    setup(context);
    while (context.nextKeyValue()) {
        map(context.getCurrentKey(), context.getCurrentValue(), context);
    }
    cleanup(context);
}

The Context.nextKeyValue(), Context.getCurrentKey() and Context.getCurrentValue() methods are wrappers for the RecordReader methods. See the source file src/mapred/org/apache/hadoop/mapreduce/MapContext.java.

So this loop executes and calls your Mapper implementation's map(K, V, Context) method.

Specifically, what else would you like to know?