Map reduce job getting stuck at map 0% reduce 0%

Question

I am running the famous wordcount example. I have a local and prod hadoop setup. The same example is working in prod, but its not working locally. Can someone tell me what s

Answer 1

If this problem is coming when using Hive queries then do check if you are joining two very big tables without leveraging partitions. Not using partitions may lead to long running full table scans and hence stuck at map 0% reduce 0%.

Answer 2

Except for hadoop tasktracker & and any other issues. Please check you code and make sure that there is no infinite loop or any other bugs. Maybe there are some bugs in your code!

Answer 3

I had the same problem and this page helped me: http://www.alexjf.net/blog/distributed-systems/hadoop-yarn-installation-definitive-guide/

Basically I solved my problem using the following 3 steps. The fact is that I had to configure much more memory I really have.

1) yarn-site.xml

• yarn.resourcemanager.hostname = hostname_of_the_master
• yarn.nodemanager.resource.memory-mb = 4000
• yarn.nodemanager.resource.cpu-vcores = 2
• yarn.scheduler.minimum-allocation-mb = 4000

2) mapred-site.xml

• yarn.app.mapreduce.am.resource.mb = 4000
• yarn.app.mapreduce.am.command-opts = -Xmx3768m
• mapreduce.map.cpu.vcores = 2
• mapreduce.reduce.cpu.vcores = 2

3) Send these files across all nodes

Answer 4

The tasktracker seems to be missing.

Try:

hadoop tasktracker &

Answer 5

In Hadoop 2.x this problem could be related to memory issues, you can see it in MapReduce in Hadoop 2.2.0 not working