How to overwrite/reuse the existing output path for Hadoop jobs again and agian

后端 未结 10 876
既然无缘
既然无缘 2021-02-12 10:29

I want to overwrite/reuse the existing output directory when I run my Hadoop job daily. Actually the output directory will store summarized output of each day\'s job run results

10条回答
  •  悲&欢浪女
    2021-02-12 11:12

    you need to add the setting in your main class:

    //Configuring the output path from the filesystem into the job
    FileOutputFormat.setOutputPath(job, new Path(args[1]));
    //auto_delete output dir
    OutputPath.getFileSystem(conf).delete(OutputPath);
    

提交回复
热议问题