[IOTDB-1850] The compaction rate limiter becomes invalid when use deserialize page compaction

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: master branch, 0.12.2-SNAPSHOT
• Component/s: Core/Compaction
• Sprint: 2021-10-DragonGate
• Assignee: Liuxuxin
• Reporter: 张凌哲
• Created: 15/Oct/21 01:42
• Updated: 03/Nov/21 02:10
• Resolved: 15/Oct/21 04:43

Description

The compaction rate limiter becomes invalid when use deserialize page compaction.

Before serialization, we need to estimate the serialized size and apply throughput from limiter.

 

Throughput = a * serializedsize

 

The throughput is used to block some time of limiter.

 

Currently, we use the getCurrentChunkSize in ChunkWriter, which is actually the serializedChunkSize, not estimated. If we haven't not flush already, it returns 0.

 

Then Throughput = a * 0 = 0, so this compaction will not block limiter, which means compaction throughput is not limited.

 

We should use estimateMaxSeriesMemSize in ChunkWriter.