[HADOOP-15711] Move branch-2 precommit/nightly test builds to java 8

• Type: Task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.10.0, 2.8.6, 2.9.3
• Assignee: Jonathan Hung
• Reporter: Jonathan Hung
• Created: 31/Aug/18 17:52
• Updated: 18/Jun/19 01:26
• Resolved: 09/Feb/19 01:20

Description

Branch-2 builds have been disabled for a while: https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86/

A test run here causes hdfs tests to hang: https://builds.apache.org/view/H-L/view/Hadoop/job/hadoop-qbt-branch2-java7-linux-x86-jhung/4/

Running hadoop-hdfs tests locally reveal some errors such as:

[ERROR] testComplexAppend2(org.apache.hadoop.hdfs.TestFileAppend2)  Time elapsed: 0.059 s  <<< ERROR!
java.lang.OutOfMemoryError: unable to create new native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Thread.java:714)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1164)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.saveFSImageInAllDirs(FSImage.java:1128)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.format(FSImage.java:174)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1172)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:403)
        at org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:234)
        at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:1080)
        at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:883)
        at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:514)
        at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:473)
        at org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend(TestFileAppend2.java:489)
        at org.apache.hadoop.hdfs.TestFileAppend2.testComplexAppend2(TestFileAppend2.java:543)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

I was able to get more tests passing locally by increasing the max user process count on my machine. But the error suggests that there's an issue in the tests themselves. Not sure if the error seen locally is the same reason as why jenkins builds are failing, I wasn't able to confirm based on the jenkins builds' lack of output.

Issue Links

• is broken by: HADOOP-15251 Backport HADOOP-13514 (surefire upgrade) to branch-2
• is broken by: HADOOP-13514 Upgrade maven surefire plugin to 2.20.1
• is related to: HADOOP-16053 Backport HADOOP-14816 to branch-2
• is related to: HADOOP-16219 [JDK8] Set minimum version of Hadoop 2 to JDK 8