[HADOOP-2053] OutOfMemoryError : Java heap space errors in hadoop 0.14

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.14.0, 0.14.1, 0.14.2
• Fix Version/s: 0.14.3
• Assignee: Arun Murthy
• Reporter: Lohit Vijaya Renu
• Created: 13/Oct/07 18:34
• Updated: 08/Jul/09 16:52
• Resolved: 15/Oct/07 17:20

Description

In recent hadoop 0.14 we are seeing few jobs where map taskf fail with java.lang.OutOfMemoryError: Java heap space problem
These were the same jobs which used to work fine with 0.13

<stack>
task_200710112103_0001_m_000015_1: java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2786)
at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
at java.io.DataOutputStream.write(DataOutputStream.java:90)
at org.apache.hadoop.io.Text.write(Text.java:243)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:340)
</stack>

Issue Links

• blocks: HADOOP-2054 Improve memory model for map-side sorts