Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8398 Hadoop2 unit test failures against 0.94
  3. HBASE-8381

TestTableInputFormatScan on Hadoop 2 fails because YARN kills our applications

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.94.6
    • Fix Version/s: 0.94.8
    • Component/s: hadoop2, mapreduce, test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Snippet from http://54.241.6.143/job/HBase-0.94-Hadoop-2/org.apache.hbase$hbase/72/testReport/junit/org.apache.hadoop.hbase.mapreduce/TestTableInputFormatScan1/testScanEmptyToEmpty/

      2013-04-16 04:12:06,097 WARN  [Container Monitor] monitor.ContainersMonitorImpl$MonitoringThread(436): Container [pid=30006,containerID=container_1366085506903_0001_01_000001] is running beyond virtual memory limits. Current usage: 160.9mb of 1.5gb physical memory used; 3.4gb of 3.1gb virtual memory used. Killing container.
      Dump of the process-tree for container_1366085506903_0001_01_000001 :
      	|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE
      	|- 30006 24535 30006 30006 (bash) 0 3 110800896 303 /bin/bash -c /usr/java/default/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.mapreduce.container.log.dir=/home/ec2-user/jenkins/workspace/HBase-0.94-Hadoop-2/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-1_1/application_1366085506903_0001/container_1366085506903_0001_01_000001 -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/home/ec2-user/jenkins/workspace/HBase-0.94-Hadoop-2/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-1_1/application_1366085506903_0001/container_1366085506903_0001_01_000001/stdout 2>/home/ec2-user/jenkins/workspace/HBase-0.94-Hadoop-2/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-1_1/application_1366085506903_0001/container_1366085506903_0001_01_000001/stderr   
      	|- 30024 30006 30006 30006 (java) 714 62 3517714432 40881 /usr/java/default/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.mapreduce.container.log.dir=/home/ec2-user/jenkins/workspace/HBase-0.94-Hadoop-2/target/org.apache.hadoop.mapred.MiniMRCluster/org.apache.hadoop.mapred.MiniMRCluster-logDir-nm-1_1/application_1366085506903_0001/container_1366085506903_0001_01_000001 -Dyarn.app.mapreduce.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 
      

      It started with this build according to my gmail search: http://54.241.6.143/job/HBase-0.94-Hadoop-2/70/

      Related to HBASE-8326 somehow, Nick Dimiduk?

        Attachments

        1. hbase-8381.patch
          1 kB
          Jonathan Hsieh

          Issue Links

            Activity

              People

              • Assignee:
                jmhsieh Jonathan Hsieh
                Reporter:
                jdcryans Jean-Daniel Cryans
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: