[HDDS-11415] Leader Execution at Leader

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Sumit Agrawal
• Reporter: Sumit Agrawal
• Created: 05/Sep/24 07:05
• Updated: 18/Oct/24 06:35

Description

Leader execution at leader provides various control capability and performance improvement and handling other challenges as faced. Few of problem focused to resolve:

1. Only execution at leader reduces failure scenarios, due to environment impacts like security token expiry, logic issues failure. Response is quick.
2. Parallel execution of request in controlled manner is possible (performance / throughput can be improved)
3. Batching of DB updates (as request handling output) will improve the number of Raft logs synch to all nodes.
4. Some code complexity as induced via DoubleBuffer can be reduced by handling transaction Index update. 
5. Many code having cache handling is very complex and keeps on reporting issues for cache over various operations, and new issues keep induced with new features.
6. Ranger can be offloaded by 2/3 times, as Ranger authentication will happen only on leader node. This will help further improvement of performance in end-to-end scenarios.

 

Sub-Tasks

• Ratis submit request refactor avoid code duplicate: Resolved
• Leader executor framework: Open
• PartialTableCache optimize for cleanup: Open
• OM WAL manual flush configuration: Open
• Prefix based locking for better concurrency: Open