Modifying the block placement strategy of HDFS

前端 未结 1 1353
后悔当初
后悔当初 2021-01-03 12:26

I would like to modify the default block placement strategy of HDFS to suit my application.
For example, I have two files say file1(128MB) and file2(128MB). Having th

相关标签:
1条回答
  • 2021-01-03 13:17

    The default behaviour of the block placement policy can be modified by extending the BlockPlacementPolicy interface and pointing the class to the dfs.block.replicator.classname property in the Hadoop configuration files.

    Hadoop operations are not tied to a particular node, this makes the Hadoop more resilient to the inherent problems in distributed computing. What is the requirement for having blocks for two files on a particular node? With the requirement known a better solution can be found.

    0 讨论(0)
提交回复
热议问题