How to fix “Task attempt_201104251139_0295_r_000006_0 failed to report status for 600 seconds.”

Question

I wrote a mapreduce job to extract some info from a dataset. The dataset is users\' rating about movies. The number of users is about 250K and the number of movies is about 300k

Answer 1

From https://issues.apache.org/jira/browse/HADOOP-1763

causes might be :

1. Tasktrackers run the maps successfully
2. Map outputs are served by jetty servers on the TTs.
3. All the reduce tasks connects to all the TT where maps are run. 
4. since there are lots of reduces wanting to connect the map output server, the jetty servers run out of threads (default 40)
5. tasktrackers continue to make periodic heartbeats to JT, so that they are not dead, but their jetty servers are (temporarily) down.