HBase
  1. HBase
  2. HBASE-10897

On master start, deadlock if refresh UI

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.99.0
    • Fix Version/s: 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Playing w/ MTTR recovery on trunk, master starting up deadlocked:

      Waiting to finish active master initialization:

      "ActiveMasterManager" daemon prio=10 tid=0x00007fafb5dc3800 nid=0x5fb5 waiting for monitor entry [0x00007faf8f57d000]
         java.lang.Thread.State: BLOCKED (on object monitor)
              at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(ConnectionManager.java:1683)
              - waiting to lock <0x000000064ab4b9a8> (a java.lang.Object)
              at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getMetaRegionLocation(ZooKeeperRegistry.java:53)
              at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1029)
              at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:989)
              at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getRegionLocation(ConnectionManager.java:830)
              at org.apache.hadoop.hbase.client.ConnectionAdapter.getRegionLocation(ConnectionAdapter.java:305)
              at org.apache.hadoop.hbase.client.RegionServerCallable.prepare(RegionServerCallable.java:77)
              at org.apache.hadoop.hbase.client.ScannerCallable.prepare(ScannerCallable.java:118)
              at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:101)
              at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:264)
              at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:169)
              at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:164)
              at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:107)
              at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:766)
              at org.apache.hadoop.hbase.catalog.MetaReader.fullScan(MetaReader.java:539)
              at org.apache.hadoop.hbase.catalog.MetaReader.fullScanOfMeta(MetaReader.java:140)
              at org.apache.hadoop.hbase.catalog.MetaMigrationConvertingToPB.isMetaTableUpdated(MetaMigrationConvertingToPB.java:164)
              at org.apache.hadoop.hbase.catalog.MetaMigrationConvertingToPB.updateMetaIfNecessary(MetaMigrationConvertingToPB.java:131)
              at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:567)
              at org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:147)
              at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1242)
              at java.lang.Thread.run(Thread.java:744)
      
      

      ... but the master servlet has the lock while trying to access master:

      "686004346@qtp-2101021459-0" daemon prio=10 tid=0x00007fafb5d2a800 nid=0x5fb1 waiting on condition [0x00007faf8f87f000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
              at java.lang.Thread.sleep(Native Method)
              at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1562)
              - locked <0x000000064ab4b9a8> (a java.lang.Object)
              at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1597)
              at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1805)
              - locked <0x000000064ab4b9a8> (a java.lang.Object)
              at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.listTables(ConnectionManager.java:2481)
              at org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:321)
              at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__userTables(MasterStatusTmplImpl.java:530)
              at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:255)
              at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:382)
              at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:372)
              at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:102)
      ...
      
      1. hbase-10897.patch
        0.7 kB
        Jimmy Xiang
      2. hbase-10897_v2.patch
        7 kB
        Jimmy Xiang
      3. hbase-10897_v3.patch
        9 kB
        Jimmy Xiang
      4. hbase-10897_v4.patch
        10 kB
        Jimmy Xiang

        Activity

        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #5077 (See https://builds.apache.org/job/HBase-TRUNK/5077/)
        HBASE-10897 On master start, deadlock if refresh UI (jxiang: rev 1586705)

        • /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #5077 (See https://builds.apache.org/job/HBase-TRUNK/5077/ ) HBASE-10897 On master start, deadlock if refresh UI (jxiang: rev 1586705) /hbase/trunk/hbase-server/src/main/jamon/org/apache/hadoop/hbase/tmpl/master/MasterStatusTmpl.jamon /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java
        Hide
        Jimmy Xiang added a comment -

        Integrated into trunk. Thanks Stack for the review.

        Show
        Jimmy Xiang added a comment - Integrated into trunk. Thanks Stack for the review.
        Hide
        stack added a comment -

        Go for it Jimmy

        Show
        stack added a comment - Go for it Jimmy
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12639610/hbase-10897_v4.patch
        against trunk revision .
        ATTACHMENT ID: 12639610

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12639610/hbase-10897_v4.patch against trunk revision . ATTACHMENT ID: 12639610 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9248//console This message is automatically generated.
        Hide
        Jimmy Xiang added a comment -

        Attached v4 that has the fixes mentioned right above.

        Show
        Jimmy Xiang added a comment - Attached v4 that has the fixes mentioned right above.
        Hide
        Jimmy Xiang added a comment -

        Is this right?
        + if (master.getMasterCoprocessorHost() == null) {
        What do CPs have to do w/ this?

        It doesn't look right. Let me do it in a different way.

        Has to be public?
        + public boolean isCatalogJanitorEnabled() {

        I will make it packaged protected.

        Show
        Jimmy Xiang added a comment - Is this right? + if (master.getMasterCoprocessorHost() == null) { What do CPs have to do w/ this? It doesn't look right. Let me do it in a different way. Has to be public? + public boolean isCatalogJanitorEnabled() { I will make it packaged protected.
        Hide
        Mikhail Antonov added a comment -

        I think it was in HBASE-10813. Shall it be reopened or new jira filed?

        Show
        Mikhail Antonov added a comment - I think it was in HBASE-10813 . Shall it be reopened or new jira filed?
        Hide
        stack added a comment -

        A commit just broke org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler ? This one seems to show up a bunch now lately.

        Show
        stack added a comment - A commit just broke org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler ? This one seems to show up a bunch now lately.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12639484/hbase-10897_v3.patch
        against trunk revision .
        ATTACHMENT ID: 12639484

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12639484/hbase-10897_v3.patch against trunk revision . ATTACHMENT ID: 12639484 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9237//console This message is automatically generated.
        Hide
        stack added a comment -

        Is this right?

        + if (master.getMasterCoprocessorHost() == null) {

        What do CPs have to do w/ this?

        Has to be public?

        + public boolean isCatalogJanitorEnabled() {

        Does + public boolean isCatalogJanitorEnabled() { signify Master is up? If so, should the test be isMasterUp? (Do we have one of these already?)

        Else its great that the UI comes up early.

        Show
        stack added a comment - Is this right? + if (master.getMasterCoprocessorHost() == null) { What do CPs have to do w/ this? Has to be public? + public boolean isCatalogJanitorEnabled() { Does + public boolean isCatalogJanitorEnabled() { signify Master is up? If so, should the test be isMasterUp? (Do we have one of these already?) Else its great that the UI comes up early.
        Hide
        Jimmy Xiang added a comment -

        Attached v3 that shows master web UI as soon as possible. Added checking before each section is available.

        Show
        Jimmy Xiang added a comment - Attached v3 that shows master web UI as soon as possible. Added checking before each section is available.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12639438/hbase-10897_v2.patch
        against trunk revision .
        ATTACHMENT ID: 12639438

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler
        org.apache.hadoop.hbase.master.TestAssignmentManager

        -1 core zombie tests. There are 1 zombie test(s):

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12639438/hbase-10897_v2.patch against trunk revision . ATTACHMENT ID: 12639438 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler org.apache.hadoop.hbase.master.TestAssignmentManager -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9231//console This message is automatically generated.
        Hide
        Jimmy Xiang added a comment -

        Attached v2 that doesn't show tables before master is initialized.

        Show
        Jimmy Xiang added a comment - Attached v2 that doesn't show tables before master is initialized.
        Hide
        Jimmy Xiang added a comment -

        With this fix, the master web UI is not available while it's finishing the initialization. Perhaps we just need to wait till meta is assigned? Let me take another look.

        Show
        Jimmy Xiang added a comment - With this fix, the master web UI is not available while it's finishing the initialization. Perhaps we just need to wait till meta is assigned? Let me take another look.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #5058 (See https://builds.apache.org/job/HBase-TRUNK/5058/)
        HBASE-10897 On master start, deadlock if refresh UI (jxiang: rev 1584091)

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #5058 (See https://builds.apache.org/job/HBase-TRUNK/5058/ ) HBASE-10897 On master start, deadlock if refresh UI (jxiang: rev 1584091) /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterStatusServlet.java
        Hide
        Jimmy Xiang added a comment -

        Committed to trunk (as-is). Thanks Stack for the review, and finding out the issue.

        Show
        Jimmy Xiang added a comment - Committed to trunk (as-is). Thanks Stack for the review, and finding out the issue.
        Hide
        stack added a comment -

        Jimmy Xiang commit it. I'm testing something else but as side effect will notice if this comes up again.

        Show
        stack added a comment - Jimmy Xiang commit it. I'm testing something else but as side effect will notice if this comes up again.
        Hide
        Jimmy Xiang added a comment -

        With this patch, I don't see the problem any more. stack, you want to give it a try?

        If the master hasn't finished the initialization, currently, we show this:

        HTTP ERROR 503
        Problem accessing /master-status. Reason:
            Master not ready
        Powered by Jetty://
        

        Should we show something like this, which looks like more friendly?

        The Master is initializing!
        
        Show
        Jimmy Xiang added a comment - With this patch, I don't see the problem any more. stack , you want to give it a try? If the master hasn't finished the initialization, currently, we show this: HTTP ERROR 503 Problem accessing /master-status. Reason: Master not ready Powered by Jetty:// Should we show something like this, which looks like more friendly? The Master is initializing!
        Hide
        stack added a comment -

        Lets try it. Good one Jimmy +1

        Show
        stack added a comment - Lets try it. Good one Jimmy +1
        Hide
        Jimmy Xiang added a comment -

        This is the patch I was playing with now.

        Show
        Jimmy Xiang added a comment - This is the patch I was playing with now.
        Hide
        Jimmy Xiang added a comment -

        I noticed this issue too. I think we should show "Master not ready" before master finishes the initialization. Let me play with a simple patch.

        Show
        Jimmy Xiang added a comment - I noticed this issue too. I think we should show "Master not ready" before master finishes the initialization. Let me play with a simple patch.
        Hide
        stack added a comment -

        We should not allow listing tables until active master Jimmy Xiang?

        Show
        stack added a comment - We should not allow listing tables until active master Jimmy Xiang ?

          People

          • Assignee:
            Jimmy Xiang
            Reporter:
            stack
          • Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development