[HBASE-20544] downstream HBaseTestingUtility fails with invalid port

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 3.0.0-alpha-1, 2.1.0, 2.0.1
• Component/s: test

Release Note

<!-- markdown -->

HBase now relies on an internal mechanism to determine when it is running a local hbase cluster meant for external interaction vs an encapsulated test. When created via the `HBaseTestingUtility`, ports for Master and RegionServer services and UIs will be set to random ports to allow for multiple parallel uses on a single machine. Normally when running a Standalone HBase Deployment (as described in the HBase Reference Guide) the ports will be picked according to the same defaults used in a full cluster set up. If you wish to instead use the random port assignment set `hbase.localcluster.assign.random.ports` to true.

• Assignee: Sean Busbey
• Reporter: Sean Busbey
• Created: 08/May/18 13:12
• Updated: 01/Aug/18 06:21
• Resolved: 14/May/18 18:05

Description

Attempting to update hbase-downstreamer to use our 2.0.0 release fails with an invalid port in the event that hbase.localcluster.assign.random.ports isn't set (or is set to false, specifically):

2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer (HRegionServer.java:<init>(631)) - Failed construction RegionServer
java.lang.IllegalArgumentException: port out of range:-1
	at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
	at java.net.InetSocketAddress.<init>(InetSocketAddress.java:224)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:1217)
	at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:1184)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:561)
	at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.<init>(MiniHBaseCluster.java:147)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
	at org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
	at org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
	at org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
	at org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:121)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
	at org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:367)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:274)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)

This is due to HBASE-15835 mistakenly setting hte REGIONSERVER_PORT to -1 rather than 0 (pick a port) or setting REGIONSERVER_INFO_PORT to -1 (which means "turn off the UI").

Ideally I think we don't need the "change what we do if default is set" behavior from HBASE-15835. It misses some other ports that will still conflict with a running HBase instance. Instead we should rely on hbase.localcluster.assign.random.ports=true as a default for HBTU.

Issue Links

• is duplicated by: HBASE-20580 Fixed the flaky TestEnableRSGroup which was caused by HBASE-20544
• is related to: HBASE-20224 Web UI is broken in standalone mode