hdfs dfs -put with overwrite?

Question

I am using

hdfs dfs -put myfile mypath

and for some files I get

put: \'myfile\': File Exists

• does

Answer 1

You can overwrite your file in hdfs using -f command.For example

hadoop fs -put -f <localfile> <hdfsDir>

OR

hadoop fs -copyFromLocal -f <localfile> <hdfsDir>

It worked fine for me. However -f command won't work in case of get or copyToLocal command. check this question

Answer 2

Just updates to this answer, in Hadoop 3.X the command a bit different

hdfs dfs -put -f /local/to/path hdfs://localhost:9870/users/XXX/folder/folder2

Answer 3

put: 'myfile': File Exists

Means,the file named "myfile" already exists in hdfs. You cannot have multiple files of the same name in hdfs

You can overwrite it using hadoop fs -put -f /path_to_local /path_to_hdfs

Answer 4

1. A file with the same name exists at the location you're trying to write to.
2. You can overwrite by specifying the -f flag.