Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15711

Move branch-2 precommit/nightly test builds to java 8

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.10.0, 2.8.6, 2.9.3
    • Component/s: None
    • Labels:
      None

      Description

      Branch-2 builds have been disabled for a while: https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/

      A test run here causes hdfs tests to hang: https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86-jhung/4/

      Running hadoop-hdfs tests locally reveal some errors such as:

      [ERROR] testComplexAppend2(org.apache.hadoop.hdfs.TestFileAppend2)  Time elapsed: 0.059 s  <<< ERROR!
      java.lang.OutOfMemoryError: unable to create new native thread
              at java.lang.Thread.start0(Native Method)
              at java.lang.Thread.start(Thread.java:714)
              at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1164)
              at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1128)
              at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:174)
              at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1172)
              at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:403)
              at org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:234)
              at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:1080)
              at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:883)
              at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:514)
              at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:473)
              at org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend(TestFileAppend2.java:489)
              at org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend2(TestFileAppend2.java:543)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      I was able to get more tests passing locally by increasing the max user process count on my machine. But the error suggests that there's an issue in the tests themselves. Not sure if the error seen locally is the same reason as why jenkins builds are failing, I wasn't able to confirm based on the jenkins builds' lack of output.

        Attachments

        1. HADOOP-15711.001.branch-2.patch
          5 kB
          Arun Suresh
        2. HADOOP-15711-branch-2.002.patch
          1.0 kB
          Jonathan Hung

        Issue Links

          Activity

            People

            • Assignee:
              jhung Jonathan Hung
              Reporter:
              jhung Jonathan Hung

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment