[HDDS-11152] OMDoubleBuffer error when handling OMRequest: cmdType: SnapshotMoveDeletedKeys

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0, 1.4.1
• Component/s: OM, Snapshot
• Labels: ozone-snapshot pull-request-available
• Target Version/s: 2.0.0, 1.4.1
• Assignee: Hemant Kumar Assign to me
• Reporter: Jyotirmoy Sinha
• Created: 12/Jul/24 06:40
• Updated: 15/Oct/24 17:47
• Resolved: 28/Aug/24 23:25

Description

OMDoubleBuffer error when handling OMRequest: cmdType: SnapshotMoveDeletedKeys

Error stacktrace - 

2024-07-11 16:39:41,419 ERROR [om136-OMDoubleBufferFlushThread]-org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer: Terminating with exit status 1: During flush to DB encountered error in OMDoubleBuffer flush thread om136-OMDoubleBufferFlushThread when handling OMRequest: cmdType: SnapshotMoveDeletedKeys
traceID: ""
success: true
status: OKjava.io.IOException: Rocks Database is closed
        at org.apache.hadoop.hdds.utils.db.RocksDatabase.acquire(RocksDatabase.java:439)
        at org.apache.hadoop.hdds.utils.db.RocksDatabase.batchWrite(RocksDatabase.java:776)
        at org.apache.hadoop.hdds.utils.db.RocksDatabase.batchWrite(RocksDatabase.java:785)
        at org.apache.hadoop.hdds.utils.db.RDBBatchOperation.commit(RDBBatchOperation.java:348)
        at org.apache.hadoop.hdds.utils.db.RDBStore.commitBatchOperation(RDBStore.java:285)
        at org.apache.hadoop.ozone.om.response.snapshot.OMSnapshotMoveDeletedKeysResponse.addToDBBatch(OMSnapshotMoveDeletedKeysResponse.java:136)
        at org.apache.hadoop.ozone.om.response.OMClientResponse.checkAndUpdateDB(OMClientResponse.java:66)
        at org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.lambda$8(OzoneManagerDoubleBuffer.java:408)
        at org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.addToBatchWithTrace(OzoneManagerDoubleBuffer.java:253)
        at org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.addToBatch(OzoneManagerDoubleBuffer.java:407)
        at org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushBatch(OzoneManagerDoubleBuffer.java:353)
        at org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushCurrentBuffer(OzoneManagerDoubleBuffer.java:328)
        at org.apache.hadoop.ozone.om.ratis.OzoneManagerDoubleBuffer.flushTransactions(OzoneManagerDoubleBuffer.java:295)
        at java.base/java.lang.Thread.run(Thread.java:834) 

Issue Links

• links to: GitHub Pull Request #7112