[HDDS-2986] MiniOzoneChaosCluster exits because of datanode shutdown

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Ozone Datanode
• Labels: MiniOzoneChaosCluster
• Assignee: Siddharth Wagle
• Reporter: Mukul Kumar Singh
• Created: 06/Feb/20 16:38
• Updated: 09/Oct/23 14:24

Description

MiniOzoneChaosCluster exits because of datanode shutdown

2020-02-06 18:50:50,760 [Datanode State Machine Thread - 0] ERROR statemachine.DatanodeStateMachine (DatanodeStateMachine.java:start(219)) - DatanodeStateMachine Shutdown due to an critical error
2020-02-06 18:50:50,772 [RatisApplyTransactionExecutor 0] INFO  interfaces.Container (KeyValueContainer.java:flushAndSyncDB(400)) - Container 30 is synced with bcsId 7490.
2020-02-06 18:50:50,774 [RatisApplyTransactionExecutor 0] INFO  interfaces.Container (KeyValueContainer.java:close(338)) - Container 30 is closed with bcsId 7490.
2020-02-06 18:50:50,774 [Datanode State Machine Thread - 0] ERROR report.ReportManager (ReportManager.java:shutdown(82)) - Failed to shutdown Report Manager
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2088)
        at java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1475)
        at org.apache.hadoop.ozone.container.common.report.ReportManager.shutdown(ReportManager.java:80)
        at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.stopDaemon(DatanodeStateMachine.java:411)
        at org.apache.hadoop.ozone.HddsDatanodeService.stop(HddsDatanodeService.java:474)
        at org.apache.hadoop.ozone.HddsDatanodeService.terminateDatanode(HddsDatanodeService.java:454)
        at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.start(DatanodeStateMachine.java:220)
        at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$startDaemon$0(DatanodeStateMachine.java:365)
        at java.lang.Thread.run(Thread.java:748)
2020-02-06 18:50:50,774 [Datanode State Machine Thread - 0] ERROR statemachine.DatanodeStateMachine (DatanodeStateMachine.java:close(272)) - Error attempting to shutdown.
java.lang.InterruptedException
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2067)
        at java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1475)
        at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.close(DatanodeStateMachine.java:264)
        at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.stopDaemon(DatanodeStateMachine.java:412)
        at org.apache.hadoop.ozone.HddsDatanodeService.stop(HddsDatanodeService.java:474)
        at org.apache.hadoop.ozone.HddsDatanodeService.terminateDatanode(HddsDatanodeService.java:454)
        at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.start(DatanodeStateMachine.java:220)
        at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$startDaemon$0(DatanodeStateMachine.java:365)
        at java.lang.Thread.run(Thread.java:748)