Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1182

Reducers fail with OutOfMemoryError while copying Map outputs

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Modifies shuffle related memory parameters to use 'long' from 'int' so that sizes greater than maximum integer size are handled correctly
    • Tags:
      OutOfMemoryError, OOM reducer

      Description

      Reducers fail while copying Map outputs with following exception

      java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1539) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1432) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1285) at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1216) ,Error:

      Reducer's memory usage keeps on increasing and ultimately exceeds -Xmx value
      I even tried with -Xmx6.5g to each reducer but it's still failing

      While looking into the reducer logs, I found that reducers were doing shuffleInMemory every time, rather than doing shuffleOnDisk

      1. M1182-1.patch
        4 kB
        Chris Douglas
      2. M1182-1v20.patch
        2 kB
        Chris Douglas
      3. M1182-0.patch
        2 kB
        Chris Douglas
      4. M1182-0v20.patch
        2 kB
        Chris Douglas
      5. HADOOP-6357.patch
        201 kB
        Chandra Prakash Bhagtani

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Chandra Prakash Bhagtani
            Reporter:
            Chandra Prakash Bhagtani
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development