[IOTDB-2381] Deadlock caused by incorrect buffer pool size counter in VirtualStorageGroupProcessor

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: master branch
• Fix Version/s: master branch, 0.12.5
• Component/s: Core/Engine
• Labels: pull-request-available
• Assignee: Eric Pai
• Reporter: Eric Pai
• Created: 13/Jan/22 07:07
• Updated: 14/Jan/22 00:59
• Resolved: 13/Jan/22 08:32

Description

See the codes in VirtualStorageGroupProcessor https://github.com/apache/iotdb/blob/3d4bd3d0b5b59e3ca2e24c2a306d14b7e8c45902/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/VirtualStorageGroupProcessor.java#L313-L328

Here the counter currentWalPoolSize += 2 before the two allocations happen. However, if an OutOfMemoryError is thrown, at least one buffer array is null, and there's at most one currentWalPoolSize -= 1 executed.

So the currentWalPoolSize will be incorrect, which may cause the thread wait in https://github.com/apache/iotdb/blob/3d4bd3d0b5b59e3ca2e24c2a306d14b7e8c45902/server/src/main/java/org/apache/iotdb/db/engine/storagegroup/VirtualStorageGroupProcessor.java#L293 forever.

And another issue, ignoring the OutOfMemoryError is a VERY BAD choice, as it's a critical error in our environment and we should let DBA know it as soon as possible.

Issue Links

• fixes: IOTDB-2320 [rel/0.12] trimTask stuck on some StorageGroup
• links to: GitHub Pull Request #4797
• links to: GitHub Pull Request #4807