[IOTDB-1020] [Distributed] Split the persist log buffer to better absorb ingestion

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: pull-request-available
• Assignee: Houliang Qi
• Reporter: Houliang Qi
• Created: 21/Nov/20 08:16
• Updated: 15/Jan/21 07:28

Description

In the current implementation, we provide only one persist raft log buffer for each SyncLogDequeSerializer, which means if the buffer is full, we will have to wait until the buffer is flushed before we can write the next log, thus creating spikes in ingestions.

So, it is beneficial to split the WAL buffer into a working one and a flushing one, and if the last flush has completed before we ran out of the current working one, we can directly swap them, and continue writes.

Issue Links

• is a child of: IOTDB-68 IoTDB supports distributed version
• links to: GitHub Pull Request #2090