[HBASE-25140] HBase test mini cluster is working only with Hadoop 2.8.0 - 3.0.3

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.2.3
• Component/s: documentation, hadoop2, test
• Assignee: Unassigned
• Reporter: Miklos Gergely
• Created: 01/Oct/20 14:31
• Updated: 01/Oct/20 15:35

Description

Running HBaseTestingUtility.startMiniCluster() on HBase 2.2.3 works only with hadoop version range 2.8.0 - 3.0.3, for example with 2.4.1 the following exception occurs:

 

21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] - Couldn't properly initialize access to HDFS internals. Please update your WAL Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more information.21:49:04,124 [RS:0;71af2d647bb3:35715] ERROR org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper [] - Couldn't properly initialize access to HDFS internals. Please update your WAL Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more information.java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.beginFileLease(long, org.apache.hadoop.hdfs.DFSOutputStream) at java.lang.Class.getDeclaredMethod(Class.java:2130) ~[?:1.8.0_242] at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.createLeaseManager(FanOutOneBlockAsyncDFSOutputHelper.java:198) ~[hbase-server-2.2.3.jar:2.2.3] at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.<clinit>(FanOutOneBlockAsyncDFSOutputHelper.java:274) [hbase-server-2.2.3.jar:2.2.3] at java.lang.Class.forName0(Native Method) ~[?:1.8.0_242] at java.lang.Class.forName(Class.java:264) [?:1.8.0_242] at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:136) [hbase-server-2.2.3.jar:2.2.3] at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:136) [hbase-server-2.2.3.jar:2.2.3] at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) [hbase-server-2.2.3.jar:2.2.3] at org.apache.hadoop.hbase.wal.WALFactory.<init>(WALFactory.java:198) [hbase-server-2.2.3.jar:2.2.3] at org.apache.hadoop.hbase.regionserver.HRegionServer.setupWALAndReplication(HRegionServer.java:1871) [hbase-server-2.2.3.jar:2.2.3] at org.apache.hadoop.hbase.regionserver.HRegionServer.handleReportForDutyResponse(HRegionServer.java:1589) [hbase-server-2.2.3.jar:2.2.3] at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.handleReportForDutyResponse(MiniHBaseCluster.java:157) [hbase-server-2.2.3-tests.jar:2.2.3] at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1001) [hbase-server-2.2.3.jar:2.2.3] at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:184) [hbase-server-2.2.3-tests.jar:2.2.3] at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:130) [hbase-server-2.2.3-tests.jar:2.2.3] at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:168) [hbase-server-2.2.3-tests.jar:2.2.3] at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_242] at javax.security.auth.Subject.doAs(Subject.java:360) [?:1.8.0_242] at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1536) [hadoop-common-2.4.1.jar:?] at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:341) [hbase-common-2.2.3.jar:2.2.3] at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:165) [hbase-server-2.2.3-tests.jar:2.2.3] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_242]

Also upon failure during maven run it would be great if the actual exception would be displayed, not just that "Master not initialized after 200000ms".

 

Issue Links

• relates to: FLINK-19447 HBaseConnectorITCase.HBaseTestingClusterAutoStarter failed with "Master not initialized after 200000ms"