HBase
  1. HBase
  2. HBASE-5529

MR test failures becuase MALLOC_ARENA_MAX is not set

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.92.0
    • Fix Version/s: 0.94.0, 0.95.0
    • Component/s: mapreduce, test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When running unit tests on CentOS 6 I get a bunch of unit test failures in mapreduce-related tests due to:
      2012-03-03 00:14:18,776 WARN [Container Monitor]
      monitor.ContainersMonitorImpl$MonitoringThread(436): Container
      [pid=21446,containerID=container_1330762435849_0002_01_000001] is
      running beyond virtual memory limits. Current usage: 223.1mb of 2.0gb
      physical memory used; 6.9gb of 4.2gb virtual memory used. Killing
      container.

      Note: this also came up in the mapreduce project. See: https://issues.apache.org/jira/browse/MAPREDUCE-3933

      Patch coming shortly

      1. HBASE-5529-trunk.patch
        0.5 kB
        Gregory Chanan
      2. HBASE-5529-to92.patch
        0.5 kB
        Gregory Chanan

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-security #132 (See https://builds.apache.org/job/HBase-TRUNK-security/132/)
          HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298574)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-security #132 (See https://builds.apache.org/job/HBase-TRUNK-security/132/ ) HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298574) Result = FAILURE larsh : Files : /hbase/trunk/pom.xml
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2674 (See https://builds.apache.org/job/HBase-TRUNK/2674/)
          HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298574)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/trunk/pom.xml
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2674 (See https://builds.apache.org/job/HBase-TRUNK/2674/ ) HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298574) Result = FAILURE larsh : Files : /hbase/trunk/pom.xml
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92 #322 (See https://builds.apache.org/job/HBase-0.92/322/)
          HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298582)

          Result = SUCCESS
          larsh :
          Files :

          • /hbase/branches/0.92/CHANGES.txt
          • /hbase/branches/0.92/pom.xml
          Show
          Hudson added a comment - Integrated in HBase-0.92 #322 (See https://builds.apache.org/job/HBase-0.92/322/ ) HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298582) Result = SUCCESS larsh : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/pom.xml
          Hide
          Hudson added a comment -

          Integrated in HBase-0.92-security #98 (See https://builds.apache.org/job/HBase-0.92-security/98/)
          HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298582)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/branches/0.92/CHANGES.txt
          • /hbase/branches/0.92/pom.xml
          Show
          Hudson added a comment - Integrated in HBase-0.92-security #98 (See https://builds.apache.org/job/HBase-0.92-security/98/ ) HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298582) Result = FAILURE larsh : Files : /hbase/branches/0.92/CHANGES.txt /hbase/branches/0.92/pom.xml
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #20 (See https://builds.apache.org/job/HBase-0.94/20/)
          HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298575)

          Result = SUCCESS
          larsh :
          Files :

          • /hbase/branches/0.94/pom.xml
          Show
          Hudson added a comment - Integrated in HBase-0.94 #20 (See https://builds.apache.org/job/HBase-0.94/20/ ) HBASE-5529 MR test failures becuase MALLOC_ARENA_MAX is not set (Gregory Chanan) (Revision 1298575) Result = SUCCESS larsh : Files : /hbase/branches/0.94/pom.xml
          Hide
          Lars Hofhansl added a comment -

          NM. Checked another jira that Stack committed yesterday. Was targeted at 0.92.1.

          Show
          Lars Hofhansl added a comment - NM. Checked another jira that Stack committed yesterday. Was targeted at 0.92.1.
          Hide
          Lars Hofhansl added a comment -

          Committed to 0.94 and trunk.
          Waiting for stack on 0.92

          Show
          Lars Hofhansl added a comment - Committed to 0.94 and trunk. Waiting for stack on 0.92
          Hide
          Lars Hofhansl added a comment -

          @Stack: Is it 0.92.1 now or 0.92.2?

          Show
          Lars Hofhansl added a comment - @Stack: Is it 0.92.1 now or 0.92.2?
          Hide
          Gregory Chanan added a comment -

          I don't think we need to add it to hbase-env.sh. The issue (at least what I've experienced) is with the mapreduce containers. So we need it in the surefire config when running unit tests, but when running the jobs normally, hadoop already has the configuration, because of HADOOP-7154.

          If we run into these errors in another area of hbase, we can add it at that time.

          Show
          Gregory Chanan added a comment - I don't think we need to add it to hbase-env.sh. The issue (at least what I've experienced) is with the mapreduce containers. So we need it in the surefire config when running unit tests, but when running the jobs normally, hadoop already has the configuration, because of HADOOP-7154 . If we run into these errors in another area of hbase, we can add it at that time.
          Hide
          Lars Hofhansl added a comment -

          If there're no concerns I'll commit tomorrow morning.

          Again, if this is a problem during test runs, it will also be a problem when running HBase so we probably want to set this (at least as comment) in hbase-env.sh.

          Show
          Lars Hofhansl added a comment - If there're no concerns I'll commit tomorrow morning. Again, if this is a problem during test runs, it will also be a problem when running HBase so we probably want to set this (at least as comment) in hbase-env.sh.
          Hide
          Lars Hofhansl added a comment -

          No need for a separately named 0.94 patch. Thanks for the link, makes perfect sense now.

          +1 on patch.

          Lastly, is this something we should add to hbase-env.sh?

          Show
          Lars Hofhansl added a comment - No need for a separately named 0.94 patch. Thanks for the link, makes perfect sense now. +1 on patch. Lastly, is this something we should add to hbase-env.sh?
          Hide
          Gregory Chanan added a comment -

          The 0.94 patch would be identical to trunk. Do you want me to reattach with a 0.94 name?

          Regarding your questions, see here: https://issues.apache.org/jira/browse/HADOOP-7154
          That is a better explanation than I gave or the linked MR jira gives. I think that addresses all your questions, but let me know if you have more.

          Show
          Gregory Chanan added a comment - The 0.94 patch would be identical to trunk. Do you want me to reattach with a 0.94 name? Regarding your questions, see here: https://issues.apache.org/jira/browse/HADOOP-7154 That is a better explanation than I gave or the linked MR jira gives. I think that addresses all your questions, but let me know if you have more.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12517201/HBASE-5529-trunk.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javadoc. The javadoc tool appears to have generated -129 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 154 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.mapreduce.TestImportTsv
          org.apache.hadoop.hbase.mapred.TestTableMapReduce
          org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
          org.apache.hadoop.hbase.master.TestSplitLogManager

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1112//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1112//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1112//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12517201/HBASE-5529-trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -129 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 154 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.master.TestSplitLogManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1112//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1112//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1112//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment -

          Interesting. Seems reasonable
          Out of personal interest: why 4? What is the default (on other distributions)? And why is CentOS special in this regard?

          Also need an 0.94 patch (which hopefully is identical to trunk version)

          Show
          Lars Hofhansl added a comment - Interesting. Seems reasonable Out of personal interest: why 4? What is the default (on other distributions)? And why is CentOS special in this regard? Also need an 0.94 patch (which hopefully is identical to trunk version)
          Hide
          Gregory Chanan added a comment -

          Attached HBASE-5529-trunk.patch

          92 patch didn't apply to trunk. This is untested, but seems straightforward.

          Show
          Gregory Chanan added a comment - Attached HBASE-5529 -trunk.patch 92 patch didn't apply to trunk. This is untested, but seems straightforward.
          Hide
          Gregory Chanan added a comment -

          Attached HBASE-5529-to92.patch

          This patch fixes the test failures I experienced when testing 0.92

          Show
          Gregory Chanan added a comment - Attached HBASE-5529 -to92.patch This patch fixes the test failures I experienced when testing 0.92

            People

            • Assignee:
              Gregory Chanan
              Reporter:
              Gregory Chanan
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development