Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-1127 Fix failing and intermittent Ozone unit tests
  3. HDDS-2986

MiniOzoneChaosCluster exits because of datanode shutdown

    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Sprint:
      HDDS BadLands

      Description

      MiniOzoneChaosCluster exits because of datanode shutdown

      2020-02-06 18:50:50,760 [Datanode State Machine Thread - 0] ERROR statemachine.DatanodeStateMachine (DatanodeStateMachine.java:start(219)) - DatanodeStateMachine Shutdown due to an critical error
      2020-02-06 18:50:50,772 [RatisApplyTransactionExecutor 0] INFO  interfaces.Container (KeyValueContainer.java:flushAndSyncDB(400)) - Container 30 is synced with bcsId 7490.
      2020-02-06 18:50:50,774 [RatisApplyTransactionExecutor 0] INFO  interfaces.Container (KeyValueContainer.java:close(338)) - Container 30 is closed with bcsId 7490.
      2020-02-06 18:50:50,774 [Datanode State Machine Thread - 0] ERROR report.ReportManager (ReportManager.java:shutdown(82)) - Failed to shutdown Report Manager
      java.lang.InterruptedException
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088)
              at java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1475)
              at org.apache.hadoop.ozone.container.common.report.ReportManager.shutdown(ReportManager.java:80)
              at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.stopDaemon(DatanodeStateMachine.java:411)
              at org.apache.hadoop.ozone.HddsDatanodeService.stop(HddsDatanodeService.java:474)
              at org.apache.hadoop.ozone.HddsDatanodeService.terminateDatanode(HddsDatanodeService.java:454)
              at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.start(DatanodeStateMachine.java:220)
              at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$startDaemon$0(DatanodeStateMachine.java:365)
              at java.lang.Thread.run(Thread.java:748)
      2020-02-06 18:50:50,774 [Datanode State Machine Thread - 0] ERROR statemachine.DatanodeStateMachine (DatanodeStateMachine.java:close(272)) - Error attempting to shutdown.
      java.lang.InterruptedException
              at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2067)
              at java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1475)
              at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.close(DatanodeStateMachine.java:264)
              at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.stopDaemon(DatanodeStateMachine.java:412)
              at org.apache.hadoop.ozone.HddsDatanodeService.stop(HddsDatanodeService.java:474)
              at org.apache.hadoop.ozone.HddsDatanodeService.terminateDatanode(HddsDatanodeService.java:454)
              at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.start(DatanodeStateMachine.java:220)
              at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$startDaemon$0(DatanodeStateMachine.java:365)
              at java.lang.Thread.run(Thread.java:748)
      

        Attachments

        1. HDDS-2986.test.patch
          4 kB
          Siddharth Wagle

          Activity

            People

            • Assignee:
              swagle Siddharth Wagle
              Reporter:
              msingh Mukul Kumar Singh
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: