Uploaded image for project: 'Phoenix'
  1. Phoenix
  2. PHOENIX-3260

MetadataRegionObserver.postOpen() can prevent region server from shutting down for a long duration

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.8.1
    • Labels:
      None

      Description

      On my local setup, I noticed that invoking ./stop-hbase.sh wasn't able to shutdown the hbase cluster for a long time. Doing a jstack revealed that the thread being created in MetadataRegionObserver.postOpen() is not assigned as daemon. This could prevent the region server hosting the SYSTEM.CATALOG table from shutting down for a long duration.

      Stacktrace of the hung thread:

      "Thread-87" prio=5 tid=0x00007ffbd699d800 nid=0x1b607 waiting on condition [0x0000000120446000]
         java.lang.Thread.State: TIMED_WAITING (sleeping)
      	at java.lang.Thread.sleep(Native Method)
      	at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:149)
      	- locked <0x00000007d1ab8140> (a org.apache.hadoop.hbase.client.RpcRetryingCaller)
      	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:833)
      	at org.apache.hadoop.hbase.client.HTable.get(HTable.java:810)
      	at org.apache.hadoop.hbase.coprocessor.CoprocessorHost$Environment$HTableWrapper.get(CoprocessorHost.java:466)
      	at org.apache.phoenix.util.UpgradeUtil.truncateStats(UpgradeUtil.java:1226)
      	at org.apache.phoenix.coprocessor.MetaDataRegionObserver$1.run(MetaDataRegionObserver.java:148)
      	at java.lang.Thread.run(Thread.java:745)
      

        Attachments

        1. PHOENIX-3260.patch
          0.8 kB
          Samarth Jain

          Activity

            People

            • Assignee:
              samarthjain Samarth Jain
              Reporter:
              samarthjain Samarth Jain
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: