HBase
  1. HBase
  2. HBASE-5441

HRegionThriftServer may not start because of a race-condition

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: Thrift
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This happens because the master is not started when ThriftServerRunner tries to create an HBaseAdmin.

      org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet
      at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1333)

      at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)
      at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
      at $Proxy8.getProtocolVersion(Unknown Source)
      at org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
      at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
      at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
      at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
      at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:649)
      at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:108)
      at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.<init>(ThriftServerRunner.java:516)
      at org.apache.hadoop.hbase.regionserver.HRegionThriftServer$HBaseHandlerRegion.<init>(HRegionThriftServer.java:104)
      at org.apache.hadoop.hbase.regionserver.HRegionThriftServer.<init>(HRegionThriftServer.java:74)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeThreads(HRegionServer.java:646)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:546)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:658)
      at java.lang.Thread.run(Thread.java:662)
      2012-02-21 16:38:18,223 INFO org.apache.hadoop.hba

        Activity

        Hide
        stack added a comment -

        Marking closed.

        Show
        stack added a comment - Marking closed.
        Hide
        Harsh J added a comment -

        Setting Fix Version to 0.95 as its available in that branch. No CHANGES.txt entry was added by this commit btw.

        Show
        Harsh J added a comment - Setting Fix Version to 0.95 as its available in that branch. No CHANGES.txt entry was added by this commit btw.
        Hide
        Phabricator added a comment -

        sc has closed the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        To: tedyu, dhruba, JIRA, sc

        Show
        Phabricator added a comment - sc has closed the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". REVISION DETAIL https://reviews.facebook.net/D1857 To: tedyu, dhruba, JIRA, sc
        Hide
        Phabricator added a comment -

        dhruba has resigned from the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        REVISION DETAIL
        https://reviews.facebook.net/D1845

        To: tedyu, JIRA, sc

        Show
        Phabricator added a comment - dhruba has resigned from the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". REVISION DETAIL https://reviews.facebook.net/D1845 To: tedyu, JIRA, sc
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2671 (See https://builds.apache.org/job/HBase-TRUNK/2671/)
        HBASE-5441 HRegionThriftServer may not start because of a race-condition (Scott Chen) (Revision 1295192)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2671 (See https://builds.apache.org/job/HBase-TRUNK/2671/ ) HBASE-5441 HRegionThriftServer may not start because of a race-condition (Scott Chen) (Revision 1295192) Result = FAILURE tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #125 (See https://builds.apache.org/job/HBase-TRUNK-security/125/)
        HBASE-5441 HRegionThriftServer may not start because of a race-condition (Scott Chen) (Revision 1295192)

        Result = FAILURE
        tedyu :
        Files :

        • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #125 (See https://builds.apache.org/job/HBase-TRUNK-security/125/ ) HBASE-5441 HRegionThriftServer may not start because of a race-condition (Scott Chen) (Revision 1295192) Result = FAILURE tedyu : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        Hide
        Scott Chen added a comment -

        Thanks for the review and commit, Ted.

        Show
        Scott Chen added a comment - Thanks for the review and commit, Ted.
        Hide
        dhruba borthakur added a comment -

        Thanks for integrating it with trunk and committing it, Ted.

        Show
        dhruba borthakur added a comment - Thanks for integrating it with trunk and committing it, Ted.
        Hide
        Ted Yu added a comment -

        Patch v5 integrated to TRUNK.

        Thanks for the patch Scott.

        Thanks for the review Dhruba.

        Show
        Ted Yu added a comment - Patch v5 integrated to TRUNK. Thanks for the patch Scott. Thanks for the review Dhruba.
        Hide
        Ted Yu added a comment -

        Patch v5 applies to TRUNK.

        Show
        Ted Yu added a comment - Patch v5 applies to TRUNK.
        Hide
        dhruba borthakur added a comment -

        This patch looks good and has been running in our clusters for about a week. Code looks good +1.

        Show
        dhruba borthakur added a comment - This patch looks good and has been running in our clusters for about a week. Code looks good +1.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12515639/HBASE-5441.D1857.4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1047//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12515639/HBASE-5441.D1857.4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1047//console This message is automatically generated.
        Hide
        Ted Yu added a comment -

        Latest test run is clean - no test regression.

        Will integrate the patch tomorrow if I don't hear objection.

        Show
        Ted Yu added a comment - Latest test run is clean - no test regression. Will integrate the patch tomorrow if I don't hear objection.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12515639/HBASE-5441.D1857.4.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 javadoc. The javadoc tool appears to have generated -136 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 151 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.replication.TestReplication
        org.apache.hadoop.hbase.mapreduce.TestImportTsv
        org.apache.hadoop.hbase.mapred.TestTableMapReduce
        org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1013//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1013//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1013//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12515639/HBASE-5441.D1857.4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -136 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 151 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.replication.TestReplication org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1013//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1013//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1013//console This message is automatically generated.
        Hide
        Phabricator added a comment -

        tedyu has commented on the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        Patch looks good.
        Will integrate after unit tests pass.

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        BRANCH
        hb-5441

        Show
        Phabricator added a comment - tedyu has commented on the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". Patch looks good. Will integrate after unit tests pass. REVISION DETAIL https://reviews.facebook.net/D1857 BRANCH hb-5441
        Hide
        Phabricator added a comment -

        sc updated the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".
        Reviewers: tedyu, dhruba, JIRA

        Synchronized on this instead of admin

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        AFFECTED FILES
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java

        Show
        Phabricator added a comment - sc updated the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". Reviewers: tedyu, dhruba, JIRA Synchronized on this instead of admin REVISION DETAIL https://reviews.facebook.net/D1857 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        Hide
        Phabricator added a comment -

        sc has commented on the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        Sorry. I synchronized on the wrong reference and I should have run the test before update the patch. I will update it soon.

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        BRANCH
        hb-5441

        Show
        Phabricator added a comment - sc has commented on the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". Sorry. I synchronized on the wrong reference and I should have run the test before update the patch. I will update it soon. REVISION DETAIL https://reviews.facebook.net/D1857 BRANCH hb-5441
        Hide
        Phabricator added a comment -

        tedyu has commented on the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        Hadoop QA reported the following in test failures:

        java.lang.NullPointerException
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getHBaseAdmin(ThriftServerRunner.java:502)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getTableNames(ThriftServerRunner.java:563)
        at org.apache.hadoop.hbase.thrift.TestThriftServer.createTestTables(TestThriftServer.java:169)
        at org.apache.hadoop.hbase.thrift.TestThriftServer.doTestTableCreateDrop(TestThriftServer.java:119)

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        BRANCH
        hb-5441

        Show
        Phabricator added a comment - tedyu has commented on the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". Hadoop QA reported the following in test failures: java.lang.NullPointerException at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getHBaseAdmin(ThriftServerRunner.java:502) at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.getTableNames(ThriftServerRunner.java:563) at org.apache.hadoop.hbase.thrift.TestThriftServer.createTestTables(TestThriftServer.java:169) at org.apache.hadoop.hbase.thrift.TestThriftServer.doTestTableCreateDrop(TestThriftServer.java:119) REVISION DETAIL https://reviews.facebook.net/D1857 BRANCH hb-5441
        Hide
        Phabricator added a comment -

        tedyu has accepted the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        BRANCH
        hb-5441

        Show
        Phabricator added a comment - tedyu has accepted the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". REVISION DETAIL https://reviews.facebook.net/D1857 BRANCH hb-5441
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12515545/HBASE-5441.D1857.3.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 javadoc. The javadoc tool appears to have generated -136 warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 152 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.thrift.TestThriftServer
        org.apache.hadoop.hbase.mapreduce.TestImportTsv
        org.apache.hadoop.hbase.mapred.TestTableMapReduce
        org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
        org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1008//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1008//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1008//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12515545/HBASE-5441.D1857.3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -136 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 152 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.thrift.TestThriftServer org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1008//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1008//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1008//console This message is automatically generated.
        Hide
        Phabricator added a comment -

        dhruba has commented on the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        Looks good to me. +1

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        Show
        Phabricator added a comment - dhruba has commented on the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". Looks good to me. +1 REVISION DETAIL https://reviews.facebook.net/D1857
        Hide
        Phabricator added a comment -

        sc has commented on the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        The reason why this works is
        "JDK5 and later extends the semantics for volatile so that the system will not allow a write of a volatile to be reordered with respect to any previous read or write, and a read of a volatile cannot be reordered with respect to any following read or write."

        It is explained in
        http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        Show
        Phabricator added a comment - sc has commented on the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". The reason why this works is "JDK5 and later extends the semantics for volatile so that the system will not allow a write of a volatile to be reordered with respect to any previous read or write, and a read of a volatile cannot be reordered with respect to any following read or write." It is explained in http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html REVISION DETAIL https://reviews.facebook.net/D1857
        Hide
        Phabricator added a comment -

        sc updated the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".
        Reviewers: tedyu, dhruba, JIRA

        Addressed the review comment from Dhruba. Thanks!

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        AFFECTED FILES
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java

        Show
        Phabricator added a comment - sc updated the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". Reviewers: tedyu, dhruba, JIRA Addressed the review comment from Dhruba. Thanks! REVISION DETAIL https://reviews.facebook.net/D1857 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        Hide
        Phabricator added a comment -

        sc has commented on the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        INLINE COMMENTS
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java:501 I see. If the reference is volatile than it is OK. I will make the change again.

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        Show
        Phabricator added a comment - sc has commented on the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java:501 I see. If the reference is volatile than it is OK. I will make the change again. REVISION DETAIL https://reviews.facebook.net/D1857
        Hide
        Phabricator added a comment -

        dhruba has commented on the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        INLINE COMMENTS
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java:501 every call will have to go via a synchronized section, will definitely slow down throughput. How about if we do the the 'volatile thing' described in http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html

        class Foo {
        private volatile Helper helper = null;
        public Helper getHelper() {
        Helper result = helper;
        if (result == null) {
        synchronized(this) {
        result = helper;
        if (result == null)

        { helper = result = new Helper(); }

        }
        }
        return result;
        }

        // other functions and members...
        }

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        Show
        Phabricator added a comment - dhruba has commented on the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java:501 every call will have to go via a synchronized section, will definitely slow down throughput. How about if we do the the 'volatile thing' described in http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html class Foo { private volatile Helper helper = null; public Helper getHelper() { Helper result = helper; if (result == null) { synchronized(this) { result = helper; if (result == null) { helper = result = new Helper(); } } } return result; } // other functions and members... } REVISION DETAIL https://reviews.facebook.net/D1857
        Hide
        Phabricator added a comment -

        sc updated the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".
        Reviewers: tedyu, dhruba, JIRA

        Address Ted's review comment. Thanks!

        REVISION DETAIL
        https://reviews.facebook.net/D1857

        AFFECTED FILES
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java

        Show
        Phabricator added a comment - sc updated the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". Reviewers: tedyu, dhruba, JIRA Address Ted's review comment. Thanks! REVISION DETAIL https://reviews.facebook.net/D1857 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        Hide
        Phabricator added a comment -

        sc has commented on the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        INLINE COMMENTS
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java:497 Yes, I will remove this. Thanks for the comment.

        Double-check locking does not actually work:
        http://en.wikipedia.org/wiki/Double-checked_locking

        I will update the patch soon.

        REVISION DETAIL
        https://reviews.facebook.net/D1845

        Show
        Phabricator added a comment - sc has commented on the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java:497 Yes, I will remove this. Thanks for the comment. Double-check locking does not actually work: http://en.wikipedia.org/wiki/Double-checked_locking I will update the patch soon. REVISION DETAIL https://reviews.facebook.net/D1845
        Hide
        Phabricator added a comment -

        tedyu has commented on the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".

        INLINE COMMENTS
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java:497 How about adding synchronized to this method and remove double checking against null ?

        REVISION DETAIL
        https://reviews.facebook.net/D1845

        Show
        Phabricator added a comment - tedyu has commented on the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java:497 How about adding synchronized to this method and remove double checking against null ? REVISION DETAIL https://reviews.facebook.net/D1845
        Hide
        Phabricator added a comment -

        sc updated the revision "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".
        Reviewers: tedyu, dhruba, JIRA

        Fix a bug

        REVISION DETAIL
        https://reviews.facebook.net/D1845

        AFFECTED FILES
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java

        Show
        Phabricator added a comment - sc updated the revision " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". Reviewers: tedyu, dhruba, JIRA Fix a bug REVISION DETAIL https://reviews.facebook.net/D1845 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java
        Hide
        Phabricator added a comment -

        sc requested code review of "HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition".
        Reviewers: tedyu, dhruba, JIRA

        HRegionThriftServer may not start because of a race-condition

        This happens because the master is not started when ThriftServerRunner tries to create an HBaseAdmin.

        org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet
        at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1333)

        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899)
        at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150)
        at $Proxy8.getProtocolVersion(Unknown Source)
        at org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183)
        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303)
        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280)
        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:649)
        at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:108)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.<init>(ThriftServerRunner.java:516)
        at org.apache.hadoop.hbase.regionserver.HRegionThriftServer$HBaseHandlerRegion.<init>(HRegionThriftServer.java:104)
        at org.apache.hadoop.hbase.regionserver.HRegionThriftServer.<init>(HRegionThriftServer.java:74)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeThreads(HRegionServer.java:646)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:546)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:658)
        at java.lang.Thread.run(Thread.java:662)
        2012-02-21 16:38:18,223 INFO org.apache.hadoop.hba

        TEST PLAN
        TBD

        REVISION DETAIL
        https://reviews.facebook.net/D1845

        AFFECTED FILES
        src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/3915/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - sc requested code review of " HBASE-5441 [jira] HRegionThriftServer may not start because of a race-condition". Reviewers: tedyu, dhruba, JIRA HRegionThriftServer may not start because of a race-condition This happens because the master is not started when ThriftServerRunner tries to create an HBaseAdmin. org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1333) at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:899) at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150) at $Proxy8.getProtocolVersion(Unknown Source) at org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:183) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:303) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:280) at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:332) at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:649) at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:108) at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.<init>(ThriftServerRunner.java:516) at org.apache.hadoop.hbase.regionserver.HRegionThriftServer$HBaseHandlerRegion.<init>(HRegionThriftServer.java:104) at org.apache.hadoop.hbase.regionserver.HRegionThriftServer.<init>(HRegionThriftServer.java:74) at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeThreads(HRegionServer.java:646) at org.apache.hadoop.hbase.regionserver.HRegionServer.preRegistrationInitialization(HRegionServer.java:546) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:658) at java.lang.Thread.run(Thread.java:662) 2012-02-21 16:38:18,223 INFO org.apache.hadoop.hba TEST PLAN TBD REVISION DETAIL https://reviews.facebook.net/D1845 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/thrift/ThriftServerRunner.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3915/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          People

          • Assignee:
            Scott Chen
            Reporter:
            Scott Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development