[HBASE-18458] Refactor TestRegionServerHostname to make it robust (Port HBASE-17922 to branch-1) - ASF JIRA

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Minor
Resolution: Fixed
Affects Version/s: 1.4.0
Fix Version/s: 1.4.0
Component/s: hadoop3
Labels:
None

Hadoop Flags:

Reviewed

Description

The TestRegionServerHostname is passing in branch-1; however, it always fails locally. Running tests individually always pass. Failing to start RS in some combination of test run indicates some resource leak.

Running org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
Tests run: 4, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 46.042 sec <<< FAILURE! - in org.apache.hadoop.hbase.regionserver.TestRegionServerHostname
testRegionServerHostnameReportedToMaster(org.apache.hadoop.hbase.regionserver.TestRegionServerHostname)  Time elapsed: 30.095 sec  <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 30000 milliseconds
	at java.lang.Thread.sleep(Native Method)
	at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:221)
	at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:445)
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:225)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:94)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1072)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1028)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:900)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:894)
	at org.apache.hadoop.hbase.regionserver.TestRegionServerHostname.testRegionServerHostnameReportedToMaster(TestRegionServerHostname.java:158)

When running the testRegionServerHostnameReportedToMaster alone or with another newly added test, the test passed without problem.
When running the testRegionServerHostnameReportedToMaster test with testInvalidRegionServerHostnameAbortsServer in the same test suite TestRegionServerHostname, the region server failed to start:

2017-07-25 15:34:24,132 FATAL [RS:0;192.168.1.7:64317] regionserver.HRegionServer(2182): ABORTING region server 192.168.1.7,64317,1501022063917: Unhandled: Failed suppression of fs shutdown hook: org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@668e0f60
java.lang.RuntimeException: Failed suppression of fs shutdown hook: org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@668e0f60
	at org.apache.hadoop.hbase.regionserver.ShutdownHook.suppressHdfsShutdownHook(ShutdownHook.java:204)
	at org.apache.hadoop.hbase.regionserver.ShutdownHook.install(ShutdownHook.java:84)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:940)
	at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:156)
	at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:108)
	at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:140)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:360)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1846)
	at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:307)
	at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:138)
	at java.lang.Thread.run(Thread.java:745)

~~HBASE-17922~~ addressed similar issue in Hadoop 3. I think this change is more robust than the one in branch-1 right now. Porting the change to branch-1 (with small modification due to code difference between branch-1 and branch-2) is a good idea.