Map reduce job getting stuck at map 0% reduce 0%

Question

I am running the famous wordcount example. I have a local and prod hadoop setup. The same example is working in prod, but its not working locally. Can someone tell me what s

Answer 1

I had the same problem and this page helped me: http://www.alexjf.net/blog/distributed-systems/hadoop-yarn-installation-definitive-guide/

Basically I solved my problem using the following 3 steps. The fact is that I had to configure much more memory I really have.

1) yarn-site.xml

• yarn.resourcemanager.hostname = hostname_of_the_master
• yarn.nodemanager.resource.memory-mb = 4000
• yarn.nodemanager.resource.cpu-vcores = 2
• yarn.scheduler.minimum-allocation-mb = 4000

2) mapred-site.xml

• yarn.app.mapreduce.am.resource.mb = 4000
• yarn.app.mapreduce.am.command-opts = -Xmx3768m
• mapreduce.map.cpu.vcores = 2
• mapreduce.reduce.cpu.vcores = 2

3) Send these files across all nodes