How to copy data from one HDFS to another HDFS?

Question

I have two HDFS setup and want to copy (not migrate or move) some tables from HDFS1 to HDFS2. How to copy data from one HDFS to another HDFS? Is it possible via Sqoop or other c

Answer 1

distcp command use for copying from one cluster to another cluster in parallel. You have to set the path for namenode of src and path for namenode of dst, internally it use mapper.

Example:

$ hadoop distcp  

there few options you can set for distcp

-m for no. of mapper for copying data this will increase speed of copying.

-atomic for auto commit the data.

-update will only update data that is in old version.

There are generic command for copying files in hadoop are -cp and -put but they are use only when the data volume is less.