问题
I have installed and executed an mapreduce program successfully in my system(Ubuntu 14.04).
I can see the output file as,
hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -ls /user/hadoopuser/MapReduceSample-output
Found 3 items
-rw-r--r-- 1 hadoopuser supergroup 0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_SUCCESS
drwxr-xr-x - hadoopuser supergroup 0 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/_logs
-rw-r--r-- 1 hadoopuser supergroup 880838 2014-07-09 16:10 /user/hadoopuser/MapReduceSample-output/part-00000
And I can open it on terminal using following command,
hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -cat /user/hadoopuser/MapReduceSample-output/part-00000
I can see the output file on terminal, but I can't see the full result because my output has large amount of lines.
So I want to open it on gedit
or nano
.
Need Solution.
回答1:
you can also use getmerge
to copy HDFS file to local system.
hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/MROutput
hadoop dfs -getmerge /path/to/HDFS /path/to/save
回答2:
instead of looking for plugin. You can add jar files from $HADOOP_INSTALL/bin in eclipse and compiler issues must be gone.
回答3:
You can't access HDFS file from local machine(system user), so that you can't open HDFS file using gedit
.
To open in gedit
you have to copy to local machine.
To do that, open terminal(Ctrl+Alt+T) and use copyToLocal
a Hadoop Shell Command to copy the output file into local machine.
Do the following,
hadoopuser@arul-PC:/usr/local/hadoop$ sudo bin/hadoop dfs -copyToLocal /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/Downloads/
Now you can open the output file using gedit
as follows,
$ sudo gedit /home/arul/Downloads/part-00000
Note :
- My HDFS username is
hadoopuser
. - You can move a file from HDFS to local machine. The Hadoop Shell Command
fs -mv
allow to move different HDFS location. - For more Hadoop Shell Commands(click here).
Update (An another option to do the same from Y-Prithvi's post)
you can also use getmerge to copy HDFS file to local system.
hadoopuser@arul-PC:/usr/local/hadoop$ bin/hadoop dfs -getmerge /user/hadoopuser/MapReduceSample-output/part-00000 /home/arul/MROutput
hadoop dfs -getmerge /path/to/HDFS /path/to/save
回答4:
Eclipse Setup for Hadoop Development
this should help
来源:https://stackoverflow.com/questions/23653645/how-to-open-hdfs-output-file-using-gedit