DiskErrorException on slave machine - Hadoop multinode

那年仲夏 提交于 2019-12-01 12:48:37

问题


I am trying to process XML files from hadoop, i got following error on invoking word-count job on XML files .

13/07/25 12:39:57 INFO mapred.JobClient: Task Id : attempt_201307251234_0001_m_000008_0, Status : FAILED
Too many fetch-failures
13/07/25 12:39:58 INFO mapred.JobClient:  map 99% reduce 0%
13/07/25 12:39:59 INFO mapred.JobClient:  map 100% reduce 0%
13/07/25 12:40:56 INFO mapred.JobClient: Task Id : attempt_201307251234_0001_m_000009_0, Status : FAILED
Too many fetch-failures
13/07/25 12:40:58 INFO mapred.JobClient:  map 99% reduce 0%
13/07/25 12:40:59 INFO mapred.JobClient:  map 100% reduce 0%
13/07/25 12:41:22 INFO mapred.JobClient:  map 100% reduce 1%
13/07/25 12:41:57 INFO mapred.JobClient: Task Id : attempt_201307251234_0001_m_000015_0, Status : FAILED
Too many fetch-failures
13/07/25 12:41:58 INFO mapred.JobClient:  map 99% reduce 1%
13/07/25 12:41:59 INFO mapred.JobClient:  map 100% reduce 1%
13/07/25 12:42:57 INFO mapred.JobClient: Task Id : attempt_201307251234_0001_m_000014_0, Status : FAILED
Too many fetch-failures
13/07/25 12:42:58 INFO mapred.JobClient:  map 99% reduce 1%
13/07/25 12:42:59 INFO mapred.JobClient:  map 100% reduce 1%
13/07/25 12:43:22 INFO mapred.JobClient:  map 100% reduce 2%

i observer following error at hadoop-hduser-tasktracker-localhost.localdomain.log file on slave machine .

2013-07-25 12:38:58,124 WARN org.apache.hadoop.mapred.TaskTracker: getMapOutput(attempt_201307251234_0001_m_000001_0,0) failed :
org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/hduser/jobcache/job_201307251234_0001/attempt_201307251234_0001_m_000001_0/output/file.out.index in any of the configured local directories
        at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)

This works fine when i ran for text files


回答1:


Looks like you have hit this issue. Either apply the patch or download the fixed version, and you should be good to go.

HTH



来源:https://stackoverflow.com/questions/17851462/diskerrorexception-on-slave-machine-hadoop-multinode

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!