Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20544

downstream HBaseTestingUtility fails with invalid port

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 3.0.0-alpha-1, 2.1.0, 2.0.1
    • Component/s: test
    • Labels:
      None
    • Release Note:
      Hide
      <!-- markdown -->

      HBase now relies on an internal mechanism to determine when it is running a local hbase cluster meant for external interaction vs an encapsulated test. When created via the `HBaseTestingUtility`, ports for Master and RegionServer services and UIs will be set to random ports to allow for multiple parallel uses on a single machine. Normally when running a Standalone HBase Deployment (as described in the HBase Reference Guide) the ports will be picked according to the same defaults used in a full cluster set up. If you wish to instead use the random port assignment set `hbase.localcluster.assign.random.ports` to true.
      Show
      <!-- markdown --> HBase now relies on an internal mechanism to determine when it is running a local hbase cluster meant for external interaction vs an encapsulated test. When created via the `HBaseTestingUtility`, ports for Master and RegionServer services and UIs will be set to random ports to allow for multiple parallel uses on a single machine. Normally when running a Standalone HBase Deployment (as described in the HBase Reference Guide) the ports will be picked according to the same defaults used in a full cluster set up. If you wish to instead use the random port assignment set `hbase.localcluster.assign.random.ports` to true.

      Description

      Attempting to update hbase-downstreamer to use our 2.0.0 release fails with an invalid port in the event that hbase.localcluster.assign.random.ports isn't set (or is set to false, specifically):

      2018-05-08 06:10:06,508 ERROR [main] regionserver.HRegionServer (HRegionServer.java:<init>(631)) - Failed construction RegionServer
      java.lang.IllegalArgumentException: port out of range:-1
      	at java.net.InetSocketAddress.checkPort(InetSocketAddress.java:143)
      	at java.net.InetSocketAddress.<init>(InetSocketAddress.java:224)
      	at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:1217)
      	at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:1184)
      	at org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:723)
      	at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:561)
      	at org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.<init>(MiniHBaseCluster.java:147)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      	at org.apache.hadoop.hbase.util.JVMClusterUtil.createRegionServerThread(JVMClusterUtil.java:86)
      	at org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:184)
      	at org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:198)
      	at org.apache.hadoop.hbase.LocalHBaseCluster$1.run(LocalHBaseCluster.java:195)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
      	at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
      	at org.apache.hadoop.hbase.LocalHBaseCluster.addRegionServer(LocalHBaseCluster.java:194)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:261)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:121)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1042)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:988)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:859)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:853)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:782)
      	at org.hbase.downstreamer.TestHBaseMiniCluster.testSpinUpMiniHBaseCluster(TestHBaseMiniCluster.java:16)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:367)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:274)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
      	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
      	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
      	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
      
      

      This is due to HBASE-15835 mistakenly setting hte REGIONSERVER_PORT to -1 rather than 0 (pick a port) or setting REGIONSERVER_INFO_PORT to -1 (which means "turn off the UI").

      Ideally I think we don't need the "change what we do if default is set" behavior from HBASE-15835. It misses some other ports that will still conflict with a running HBase instance. Instead we should rely on hbase.localcluster.assign.random.ports=true as a default for HBTU.

        Attachments

        1. HBASE-20544.0.patch
          15 kB
          Sean Busbey
        2. HBASE-20544.1.patch
          17 kB
          Sean Busbey
        3. HBASE-20544.2.patch
          23 kB
          Sean Busbey
        4. HBASE-20544.addendum.0.patch
          1 kB
          Sean Busbey

          Issue Links

            Activity

              People

              • Assignee:
                busbey Sean Busbey
                Reporter:
                busbey Sean Busbey
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: