How to open HDFS output file using gedit?

心不动则不痛 提交于 2021-02-19 06:35:17

问题


I have installed and executed an mapreduce program successfully in my system(Ubuntu 14.04).

I can see the output file as,

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -ls /user/hadoopuser/MapReduceSample-output
Found 3 items
-rw-r--r--   1 hadoopuser supergroup          0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_SUCCESS
drwxr-xr-x   - hadoopuser supergroup          0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_logs
-rw-r--r--   1 hadoopuser supergroup     880838 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/part-00000

And I can open it on terminal using following command,

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -cat /user/hadoopuser/MapReduceSample-output/part-00000

I can see the output file on terminal, but I can't see the full result because my output has large amount of lines.

So I want to open it on gedit or nano.

Need Solution.


回答1:


you can also use getmerge to copy HDFS file to local system.

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/MROutput

hadoop dfs -getmerge /path/to/HDFS /path/to/save




回答2:


instead of looking for plugin. You can add jar files from $HADOOP_INSTALL/bin in eclipse and compiler issues must be gone.




回答3:


You can't access HDFS file from local machine(system user), so that you can't open HDFS file using gedit.

To open in gedit you have to copy to local machine.

To do that, open terminal(Ctrl+Alt+T) and use copyToLocal a Hadoop Shell Command to copy the output file into local machine.

Do the following,

hadoopuser@arul-PC:/usr/local/hadoop$ sudo bin/hadoop dfs -copyToLocal /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/Downloads/

Now you can open the output file using gedit as follows,

$ sudo gedit /home/arul/Downloads/part-00000

Note :

  1. My HDFS username is hadoopuser.
  2. You can move a file from HDFS to local machine. The Hadoop Shell Command fs -mv allow to move different HDFS location.
  3. For more Hadoop Shell Commands(click here).

Update (An another option to do the same from Y-Prithvi's post)

you can also use getmerge to copy HDFS file to local system.

hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/MROutput

hadoop dfs -getmerge /path/to/HDFS /path/to/save



回答4:


Eclipse Setup for Hadoop Development

this should help



来源:https://stackoverflow.com/questions/23653645/how-to-open-hdfs-output-file-using-gedit

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!