Why does “hadoop fs -mkdir” fail with Permission Denied?

Question

I am using Cloudera on a VM machine that I am playing around with. Unfortunately I am having issues copying data to the HDFS, I am getting the following:

[cl         


        

Answer 1

I resolved the issue by creating a supergroup in /etc/group and updated the user logins on it. I mean user should be part of HDFS supergroup to have access to write on HDFS.

$vi /etc/group

supergroup:x:30000:root

Later was able to write on HDFS. Hope it helps

Answer 2

export HADOOP_USER_NAME=hdfs

Try running this command and then make a directory, it worked for me like a charm.

Answer 3

This is because you dont have enough permission to create directory in hdfs. Try running this as sudo:

sudo -u hdfs hadoop fs -mkdir -p /user/samplefolder

But this is not recommended because it compromises security.

Answer 4

Using mkdir in hadoop needs the "hadoop file permissions". From your example you can see that hdfs is a user that has permissions to create folders. So if you run:

sudo -u hdfs hadoop fs -mkdir /import

then the import folder will be created. If you want to change the owner of this folder run:

sudo -u hdfs hadoop fs -chown new_user /import

Now the new_user can manipulate files inside the import folder

Answer 5

When you execute the above command, if hdfs home directory(/user/cloudera) is not there then that directory will be created first then the directory input will be created under /user/cloudera

For giving permission for cloudera user to create it's own directory, you got to give permission. hdfs user is the admin user in hdfs switch to hdfs then execute the following command

[hdfs@localhost~]$ hadoop fs -mkdir /user/cloudera ; hadoop fs -chmod 777  /user/cloudera

Or

if you are not too concerned about hdfs security you disable hdfs permission by setting the below property to false in hdfs-site.xml

<property>
<name>dfs.permissions.enabled</name>
<value>false</value>
</property>

after setting this property to false hdfs needs to be restarted.

Answer 6

In cloudera manager, you can change the settings: hdfs->configuration->view&edit, uncheck the Check HDFS Permissions dfs.permissions and restart the hdfs.