HBase
  1. HBase
  2. HBASE-8148

Allow IPC to bind on a specific address

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.94.7, 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      hbase.master.ipc.address and hbase.regionserver.ipc.address can now be used to have HBase bind on specific IP addresses, 0.0.0.0 being the main use case. The default behavior still relies on hbase.regionserver.dns.interface/nameserver.

      Description

      Sometimes it can be useful to have the master and region servers bind on a specific address, mainly 0.0.0.0. Adding this shouldn't change the default behavior.

      1. HBASE-8148-0.94.patch
        3 kB
        Jean-Daniel Cryans
      2. HBASE-8148.patch
        4 kB
        Jean-Daniel Cryans

        Activity

        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security-on-Hadoop-23 #13 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/13/)
        HBASE-8148 Allow IPC to bind on a specific address (Revision 1459590)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #13 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/13/ ) HBASE-8148 Allow IPC to bind on a specific address (Revision 1459590) Result = FAILURE jdcryans : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94-security #129 (See https://builds.apache.org/job/HBase-0.94-security/129/)
        HBASE-8148 Allow IPC to bind on a specific address (Revision 1459590)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - Integrated in HBase-0.94-security #129 (See https://builds.apache.org/job/HBase-0.94-security/129/ ) HBASE-8148 Allow IPC to bind on a specific address (Revision 1459590) Result = FAILURE jdcryans : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95-on-hadoop2 #37 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/37/)
        HBASE-8148 Allow IPC to bind on a specific address (Revision 1459598)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - Integrated in hbase-0.95-on-hadoop2 #37 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/37/ ) HBASE-8148 Allow IPC to bind on a specific address (Revision 1459598) Result = FAILURE jdcryans : Files : /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #459 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/459/)
        HBASE-8148 Allow IPC to bind on a specific address (Revision 1459589)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #459 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/459/ ) HBASE-8148 Allow IPC to bind on a specific address (Revision 1459589) Result = FAILURE jdcryans : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Hudson added a comment -

        Integrated in HBase-0.94 #920 (See https://builds.apache.org/job/HBase-0.94/920/)
        HBASE-8148 Allow IPC to bind on a specific address (Revision 1459590)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Show
        Hudson added a comment - Integrated in HBase-0.94 #920 (See https://builds.apache.org/job/HBase-0.94/920/ ) HBASE-8148 Allow IPC to bind on a specific address (Revision 1459590) Result = FAILURE jdcryans : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        Hide
        Jean-Daniel Cryans added a comment -

        Committed to 0.94/5/6. Thanks Stack for your feedback.

        Show
        Jean-Daniel Cryans added a comment - Committed to 0.94/5/6. Thanks Stack for your feedback.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574667/HBASE-8148-0.94.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4926//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574667/HBASE-8148-0.94.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4926//console This message is automatically generated.
        Hide
        Jean-Daniel Cryans added a comment -

        +1 (Was more asking about regionservers showing in master but thinking on how this works – master tells regionserver what name to use – I can't see how this patch would mess stuff up).

        On the hostname side of things, the master remains unchanged and for the region server it's like you say yes, master tells the RS to use whatever it sees.

        Would suggest trying to get this into 0.94.

        Patch attached, I ran it on a single node to see if it works ok and it does (also checked lsof).

        Show
        Jean-Daniel Cryans added a comment - +1 (Was more asking about regionservers showing in master but thinking on how this works – master tells regionserver what name to use – I can't see how this patch would mess stuff up). On the hostname side of things, the master remains unchanged and for the region server it's like you say yes, master tells the RS to use whatever it sees. Would suggest trying to get this into 0.94. Patch attached, I ran it on a single node to see if it works ok and it does (also checked lsof).
        Hide
        stack added a comment -

        +1 (Was more asking about regionservers showing in master but thinking on how this works – master tells regionserver what name to use – I can't see how this patch would mess stuff up).

        Would suggest trying to get this into 0.94.

        Show
        stack added a comment - +1 (Was more asking about regionservers showing in master but thinking on how this works – master tells regionserver what name to use – I can't see how this patch would mess stuff up). Would suggest trying to get this into 0.94.
        Hide
        Jean-Daniel Cryans added a comment -

        lsof shows this with the config set to 0.0.0.0 (standalone so ports are kinda random):

        java 87639 jdcryans 99u IPv6 0x5de58c7d72297553 0t0 TCP *:58383 (LISTEN)

        versus without the config:

        java 87867 jdcryans 99u IPv6 0x5de58c7d669d6d93 0t0 TCP 192.168.1.120:58698 (LISTEN)

        Show
        Jean-Daniel Cryans added a comment - lsof shows this with the config set to 0.0.0.0 (standalone so ports are kinda random): java 87639 jdcryans 99u IPv6 0x5de58c7d72297553 0t0 TCP *:58383 (LISTEN) versus without the config: java 87867 jdcryans 99u IPv6 0x5de58c7d669d6d93 0t0 TCP 192.168.1.120:58698 (LISTEN)
        Hide
        stack added a comment -

        Ok. Leave as is.

        Does it work?

        Show
        stack added a comment - Ok. Leave as is. Does it work?
        Hide
        Jean-Daniel Cryans added a comment -

        Should "hbase.master.ipc.address" bit h.m.ipc.bind.address ?

        Thanks for looking at the patch. I tried to stay close to what HDFS has such as "dfs.datanode.ipc.address". Putting "bind" in there makes it clearer but then it's different from what HDFS does. Either work for me.

        Show
        Jean-Daniel Cryans added a comment - Should "hbase.master.ipc.address" bit h.m.ipc.bind.address ? Thanks for looking at the patch. I tried to stay close to what HDFS has such as "dfs.datanode.ipc.address". Putting "bind" in there makes it clearer but then it's different from what HDFS does. Either work for me.
        Hide
        stack added a comment -

        Should "hbase.master.ipc.address" bit h.m.ipc.bind.address ?

        It works?

        Show
        stack added a comment - Should "hbase.master.ipc.address" bit h.m.ipc.bind.address ? It works?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12574458/HBASE-8148.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.master.TestMasterFailover
        org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12574458/HBASE-8148.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.master.TestMasterFailover org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/4901//console This message is automatically generated.
        Hide
        Jean-Daniel Cryans added a comment -

        Patch that does the work.

        For the RS, it simply uses hbase.regionserver.ipc.address if it's available to bind.

        For the master, the ServerName cannot be set like we do for the RS when it reports for duty so I keep both the hostname that's normally set and the bind address if configured so that the Master doesn't report as being "0.0.0.0".

        Show
        Jean-Daniel Cryans added a comment - Patch that does the work. For the RS, it simply uses hbase.regionserver.ipc.address if it's available to bind. For the master, the ServerName cannot be set like we do for the RS when it reports for duty so I keep both the hostname that's normally set and the bind address if configured so that the Master doesn't report as being "0.0.0.0".

          People

          • Assignee:
            Jean-Daniel Cryans
            Reporter:
            Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development