Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8912

[0.94] AssignmentManager throws IllegalStateException from PENDING_OPEN to OFFLINE

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.16
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      AM throws this exception which subsequently causes the master to abort:

      java.lang.IllegalStateException: Unexpected state : testRetrying,jjj,1372891751115.9b828792311001062a5ff4b1038fe33b. state=PENDING_OPEN, ts=1372891751912, server=hemera.apache.org,39064,1372891746132 .. Cannot transit it to OFFLINE.
      	at org.apache.hadoop.hbase.master.AssignmentManager.setOfflineInZooKeeper(AssignmentManager.java:1879)
      	at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1688)
      	at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1424)
      	at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1399)
      	at org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1394)
      	at org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:105)
      	at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:175)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
      	at java.lang.Thread.run(Thread.java:662)
      

      This exception trace is from the failing test TestMetaReaderEditor which is failing pretty frequently, but looking at the test code, I think this is not a test-only issue, but affects the main code path.

      https://builds.apache.org/job/HBase-0.94/1036/testReport/junit/org.apache.hadoop.hbase.catalog/TestMetaReaderEditor/testRetrying/

        Attachments

        1. HBase-0.94 #1036 test - testRetrying [Jenkins].html
          978 kB
          Enis Soztutar
        2. log.txt
          36 kB
          Jean-Marc Spaggiari
        3. 8912-0.94.txt
          0.6 kB
          Lars Hofhansl
        4. org.apache.hadoop.hbase.catalog.TestMetaReaderEditor-output.txt
          1.17 MB
          Lars Hofhansl
        5. 8912-0.94-alt2.txt
          0.8 kB
          Lars Hofhansl
        6. 8912-fix-race.txt
          2 kB
          Lars Hofhansl
        7. HBASE-8912.patch
          18 kB
          ramkrishna.s.vasudevan

          Activity

            People

            • Assignee:
              lhofhansl Lars Hofhansl
              Reporter:
              enis Enis Soztutar
            • Votes:
              4 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: