Hadoop Hive unable to move source to destination

问题

I am trying to use Hive 1.2.0 over Hadoop 2.6.0. I have created an employee table. However, when I run the following query:

hive> load data local inpath '/home/abc/employeedetails' into table employee;

I get the following error:

Failed with exception Unable to move source file:/home/abc/employeedetails to destination hdfs://localhost:9000/user/hive/warehouse/employee/employeedetails_copy_1
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask

What wrong am I doing here? Are there any specific permissions that I need to set? Thanks in advance!

回答1:

As mentioned by Rio, the issue involved lack of permissions to load data into hive table. I figures out the following command solves my problems:

hadoop fs -chmod g+w /user/hive/warehouse

回答2:

See the permission for the HDFS directory:

hdfs dfs -ls /user/hive/warehouse/employee/employeedetails_copy_1

Seems like you may not have permission to load data into hive table.

回答3:

The error might be due to permission issue on local filesystem.

Change the permission for local filesystem:

sudo chmod -R 777 /home/abc/employeedetails

Now, run:

hive> load data local inpath '/home/abc/employeedetails' into table employee;

回答4:

If we face same error After running the above command in distributed mode, we can try the the below cammand in all super users of all nodes. sudo usermod -a -G hdfs yarn Note:we get this error after restart the all the services of YARN(in AMBARI).My problem was resolved.This is admin command better to care when you are running.

回答5:

I meet the same problems and search it two days .Finally I find the reason is that datenode start a moment and shut down.

solve steps:

hadoop fs -chmod -R 777 /home/abc/employeedetails
hadoop fs -chmod -R 777 /user/hive/warehouse/employee/employeedetails_copy_1
vi hdfs-site.xml and add follow infos :

dfs.permissions.enabled false
hdfs --daemon start datanode
vi hdfs-site.xml #find the location of 'dfs.datanode.data.dir'and'dfs.namenode.name.dir'.If it is the same location ,you must change it ,this is why I can't start datanode reason.
follow 'dfs.datanode.data.dir'/data/current edit the VERSION and copy clusterID to 'dfs.namenode.name.dir'/data/current clusterID of VERSION。
start-all.sh
if above it is unsolved , to be careful to follow below steps because of the safe of data ,but I already solve the problem because follow below steps.
stop-all.sh
delete the data folder of 'dfs.datanode.data.dir' and the data folder of 'dfs.namenode.name.dir' and tmp folder.
hdfs namenode -format
start-all.sh
solve the problem

maybe you will meet other problem like this.

problems:

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot create directory /opt/hive/tmp/root/1be8676a-56ac-47aa-ab1c-aa63b21ce1fc. Name node is in safe mode

methods: hdfs dfsadmin -safemode leave

来源：https://stackoverflow.com/questions/30483296/hadoop-hive-unable-to-move-source-to-destination

标签

java

Hadoop

Hive

execution