Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4379

Node Manager throws java.lang.OutOfMemoryError: Java heap space due to org.apache.hadoop.fs.LocalDirAllocator.contexts

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.3, 2.0.0-alpha, 3.0.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mrv2, nodemanager
    • Labels:
      None

      Description

      Exception in thread "Container Monitor" java.lang.OutOfMemoryError: Java heap space
      	at java.io.BufferedReader.<init>(BufferedReader.java:80)
      	at java.io.BufferedReader.<init>(BufferedReader.java:91)
      	at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:410)
      	at org.apache.hadoop.yarn.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:171)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl$MonitoringThread.run(ContainersMonitorImpl.java:389)
      	Exception in thread "LocalizerRunner for container_1340690914008_10890_01_000003" java.lang.OutOfMemoryError: Java heap space
      	at java.util.Arrays.copyOfRange(Arrays.java:3209)
      	at java.lang.String.<init>(String.java:215)
      	at com.sun.org.apache.xerces.internal.xni.XMLString.toString(XMLString.java:185)
      	at com.sun.org.apache.xerces.internal.parsers.AbstractDOMParser.characters(AbstractDOMParser.java:1188)
      	at com.sun.org.apache.xerces.internal.xinclude.XIncludeHandler.characters(XIncludeHandler.java:1084)
      	at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:464)
      	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
      	at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
      	at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
      	at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:235)
      	at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:284)
      	at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:180)
      	at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1738)
      	at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1689)
      	at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1635)
      	at org.apache.hadoop.conf.Configuration.set(Configuration.java:722)
      	at org.apache.hadoop.conf.Configuration.setStrings(Configuration.java:1300)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.initDirs(ContainerLocalizer.java:375)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ContainerLocalizer.runLocalization(ContainerLocalizer.java:127)
      	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.startLocalizer(DefaultContainerExecutor.java:103)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.run(ResourceLocalizationService.java:862)
      
      1. MAPREDUCE-4379.patch
        4 kB
        Devaraj K
      2. MAPREDUCE-4379-1.patch
        4 kB
        Devaraj K

        Activity

        Devaraj K created issue -
        Robert Joseph Evans made changes -
        Field Original Value New Value
        Affects Version/s 0.23.3 [ 12320060 ]
        Target Version/s 0.23.3 [ 12320060 ]
        Devaraj K made changes -
        Priority Critical [ 2 ] Blocker [ 1 ]
        Devaraj K made changes -
        Attachment MAPREDUCE-4379.patch [ 12533943 ]
        Devaraj K made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Devaraj K made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Devaraj K made changes -
        Attachment MAPREDUCE-4379-1.patch [ 12535334 ]
        Devaraj K made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Harsh J made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Target Version/s 0.23.3 [ 12320060 ]
        Fix Version/s 0.23.3 [ 12320060 ]
        Fix Version/s 2.0.1-alpha [ 12321442 ]
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Fix Version/s 2.0.2-alpha [ 12322471 ]
        Fix Version/s 2.1.0-alpha [ 12321442 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Devaraj K
            Reporter:
            Devaraj K
          • Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development