[HADOOP-10042] Heap space error during copy from maptask to reduce task

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 1.2.1
• Fix Version/s: 1.2.1
• Component/s: conf

Environment

Ubuntu cluster

• Assignee: Unassigned
• Reporter: Dieter De Witte
• Created: 11/Oct/13 11:59
• Updated: 12/Oct/13 07:35
• Resolved: 11/Oct/13 15:40

Description

http://stackoverflow.com/questions/19298357/out-of-memory-error-in-mapreduce-shuffle-phase

I've described the problem on stackoverflow as well. It contains a link to another JIRA: http://hadoop-common.472056.n3.nabble.com/Shuffle-In-Memory-OutOfMemoryError-td433197.html

My errors are completely the same: out of memory error when mapred.job.shuffle.input.buffer.percent = 0.7, the program does work when I put it to 0.2, does this mean the original JIRA was not resolved?

Does anybody have an idea whether this is a mapreduce issue or is it a misconfiguration from my part?