[IOTDB-1163] Optimize the insertion speed of insertRecords interface in Session

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Resolved
• Fix Version/s: 0.12.0
• Labels: pull-request-available
• Assignee: Houliang Qi
• Reporter: Houliang Qi
• Created: 20/Feb/21 03:54
• Updated: 12/Mar/21 08:17
• Resolved: 26/Feb/21 10:00

Description

At present, the insertRecords interface generates an InsertRowPlan for each record of data. In this way, a raft log will be generated in the cluster version. Because every raft log will be locked, it will bring a lot of overhead.

At the same time, if some records belong to the same raft replication group, they can be sent to other nodes of this raft group as a raft log, which will reduce the transmission of RPC.

For example, 10 records belong to the same raft replication group. If the 10 records are regarded as a raft log, one node needs one RPC. If each log is regarded as a raft log, it needs 10 RPCs.

 

And we use iotdb-benchmark to test the insertRecords interface, we can see that, after optimization, the performance is improved by 13 times.

Issue Links

• is related to: IOTDB-1099 Optimize insertablets logic in cluster module
• links to: GitHub Pull Request #2698