[HBASE-22490] Nightly client integration test fails with hadoop-3

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.2.0, 2.3.0, 2.1.6
• Component/s: build
• Assignee: Guanghao Zhang
• Reporter: Duo Zhang
• Created: 29/May/19 03:53
• Updated: 01/Jun/19 00:24
• Resolved: 01/Jun/19 00:24

Description

2019-05-28 15:27:10,107 WARN common.Storage: Storage directory /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/output-integration/hadoop-3/target/test/data/dfs/name-0-1 does not exist
2019-05-28 15:27:10,109 WARN namenode.FSNamesystem: Encountered exception loading fsimage
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /home/jenkins/jenkins-slave/workspace/HBase_Nightly_master/output-integration/hadoop-3/target/test/data/dfs/name-0-1 is in an inconsistent state: storage directory does not exist or is not accessible.
	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:376)
	at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:227)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1074)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:706)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:628)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:690)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:919)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:892)
	at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1622)
	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNode(MiniDFSCluster.java:1305)
	at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:1074)
	at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:949)
	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:881)
	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:514)
	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:473)
	at org.apache.hadoop.mapreduce.MiniHadoopClusterManager.start(MiniHadoopClusterManager.java:160)
	at org.apache.hadoop.mapreduce.MiniHadoopClusterManager.run(MiniHadoopClusterManager.java:132)
	at org.apache.hadoop.mapreduce.MiniHadoopClusterManager.main(MiniHadoopClusterManager.java:320)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
	at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
	at org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:139)
	at org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:147)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:244)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:158)

Issue Links

• is broken by: HADOOP-14970 MiniHadoopClusterManager doesn't respect lack of format option
• relates to: HBASE-22505 Nightly jobs for branch-2.1+ are always timeout
• links to: GitHub Pull Request #265