[IOTDB-900] [Distributed] Tree-structured log forwarding within a raft group.

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Core/Cluster
• Assignee: Unassigned
• Reporter: Tian Jiang
• Created: 21/Sep/20 07:32
• Updated: 21/Sep/20 07:32

Description

In a raft group, the leader takes the responsibility to replicate logs to its followers. When the number of replications is large, the network overhead of sending log replications to other nodes is significantly high, as the leader may need to send every follower a replication for each log. When there are N nodes in the group, the leader suffers N-1 times more network traffic compared with the stand-alone version.

This could be relieved by adopting a tree-like forwarding schema. Taking N=3 for an example, the leader must send log replications to both followers, denoted as 1 ->

{2, 3}

. But we could let the followers send the logs to each other, making the procedure 1 -> 2 -> 3, so the network traffic in leader may reduce to half and even less when N becomes larger.

The challenges are three folds: first, the leader must know whom to send the initial log replications, so the leader will receive agreements from the quorum the fastest; second, each follower must know whom they should further forward to so that no two nodes send replications to the same node, which wastes bandwidth; last, as some nodes do not receive requests directly from the leader, so when they accept the log, they must inform the leader properly so the leader will not perform meaningless catch-up.

Issue Links

• is a child of: IOTDB-68 IoTDB supports distributed version