Writing output to different folders hadoop

后端 未结 4 1271
暖寄归人
暖寄归人 2021-01-14 02:10
  1. I want to write two different types of output from the same reducer, into two different directories.

I am able to use multipleoutputs feature in hadoop

4条回答
  •  余生分开走
    2021-01-14 02:53

    Yes you can specify that a input format only processes certain files:

    FileInputFormat.setInputPaths(job, "/path/to/folder/testfile*");
    

    If you do amend the code, remember the _SUCCESS file should be written to both folders upon successful job completion - while this isn't a requirement, it is a machanism by which someone can determine if the output in that folder is complete, and not 'truncated' because of an error.

提交回复
热议问题