hadoop getmerge to another machine

后端 未结 1 1452

Is it possible to store the output of the hadoop dfs -getmerge command to another machine?

The reason is that there is no enough space in my local machi

1条回答
  •  礼貌的吻别
    2021-01-15 02:07

    This will do exactly what you need:

    hadoop fs -cat /user/hduser/Job-output/* | ssh user@remotehost.com "cat >mergedOutput.txt"
    

    fs -cat will read all files in sequence and output them to stdout.

    ssh will pass them to a file on remote machine (note that scp will not accept stdin as input)

    0 讨论(0)
提交回复
热议问题