[HDDS-3240] Improve write efficiency by creating container in parallel.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Labels: pull-request-available
• Assignee: runzhiwang
• Reporter: runzhiwang
• Created: 20/Mar/20 11:51
• Updated: 02/Apr/20 11:33
• Resolved: 02/Apr/20 11:33

Description

Now follower cannot create container until leader finish creating container. But follower and leader can create container in parallel rather than in sequential.
Why leader and follower create container in sequential now:
1. From the code, the future thread do getCachedStateMachineData in readStateMachineData and the future thread do createContainer in writeStateMachineData are the same thread. Because writeStateMachineData called before readStateMachineData. So leader must wait createContainer finish then getCachedStateMachineData and append logs to the follower, so leader and follower are not independent in createContainer, follower must wait leader finish createContainer.
2. From the jaeger UI, you can also see follower create container after leader finishing it currently.

How to improve it:
I think this order can be improved by distinguishing the thread used by getCachedStateMachineData and createContainer , and data = readStateMachineData(requestProto, term, logIndex) use same thread with createContainer . If stateMachineDataCache.get(logIndex) does not return null, leader can get stateMachineData from cache and need not wait createContainer finish, thus leader and follower can be independent. But if it return null, leader must finish createContainer and then apennd logs to the follower, so I think data = readStateMachineData(requestProto, term, logIndex) should use the same thread with createContainer rather than the whole getCachedStateMachineData.

Issue Links

• links to: GitHub Pull Request #734