Accessing files in hadoop distributed cache

前端 未结 4 2038
夕颜
夕颜 2021-02-06 04:42

I want to use the distributed cache to allow my mappers to access data. In main, I\'m using the command

DistributedCache.addCacheFile(new URI(\"/user/peter/cac         


        
4条回答
  •  傲寒
    傲寒 (楼主)
    2021-02-06 05:43

    Once the Job is assigned to with a configuration object, ie Configuration conf = new Configuration();

    Job job = new Job(conf, "wordcount");
    

    And then if deal with attributes of conf as shown below, eg

    conf.set("demiliter","|");
    

    or

    DistributedCache.addCacheFile(new URI("/user/peter/cacheFile/testCache1"), conf);
    

    Such changes would not be reflected in a pseudo cluster or cluster how ever it would work with local environment.

提交回复
热议问题