[HDDS-9392] Streaming write pipeline should pick the nearest datanode as primary node

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.4.0
• Component/s: OM, Ozone Client, SCM
• Labels: pull-request-available
• Assignee: Ivan Andika
• Reporter: Ivan Andika
• Created: 05/Oct/23 08:07
• Updated: 10/Dec/23 02:40
• Resolved: 10/Dec/23 01:40

Description

In `BlockDataStreamOutput#setupStream`, if pipeline mode is enabled, we pass the Pipeline object to the `RatisHelper#getRoutingTable`.

In `RatisHelper#getRoutingTable`, we pick the first datanode in the pipeline as the primary, which from my understanding should be the Datanode closest to the client. However, I'm unable to find the logic that sort the datanodes in the pipeline.

From what I understand, when we open the stream key and request SCM to allocate the block (`OMKeyRequest#allocateBlock`), we need to also pass the clientMachineString so that so that nearest datanodes will be used instead. This should improve the performance of the stream pipeline mode.

Issue Links

• relates to: HDDS-5486 Streaming supports writing in Pipline mode
• relates to: HDDS-9767 Support passing client address to SCM during write
• links to: GitHub Pull Request #5414