Accessing files in hadoop distributed cache

Question

I want to use the distributed cache to allow my mappers to access data. In main, I\'m using the command

DistributedCache.addCacheFile(new URI(\"/user/peter/cac         


        

Answer 1

This version of code ( which is slightly different from the above mentioned constructs) has always worked for me.

//in main(String [] args)
Job job = new Job(conf,"Word Count"); 
...
DistributedCache.addCacheFile(new URI(/user/peter/cacheFile/testCache1), job.getConfiguration());

I didnt see the complete setup() function in Mapper code

public void setup(Context context) throws IOException, InterruptedException {

    Configuration conf = context.getConfiguration();
    FileSystem fs = FileSystem.getLocal(conf);

    Path[] dataFile = DistributedCache.getLocalCacheFiles(conf);

    // [0] because we added just one file.
    BufferedReader cacheReader = new BufferedReader(new InputStreamReader(fs.open(dataFile[0])));
    // now one can use BufferedReader's readLine() to read data

}