[HDDS-2010] PipelineID management for multi-raft, in SCM or in datanode?

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Resolved
• Fix Version/s: 1.4.0
• Component/s: Ozone Datanode
• Labels: TriagePending
• Target Version/s: 1.4.0
• Assignee: Li Cheng
• Reporter: Li Cheng
• Created: 22/Aug/19 08:06
• Updated: 27/Mar/23 09:46
• Resolved: 27/Mar/23 09:46

Description

With the intention to support multi-raft, I wanna bring up a question on how the pipeline unique ids be managed. Since every datanode  can be member in multiple raft pipelines, the pipeline ids need to be persisted with the datanode for recovery purpose (we can talk about recovery later). Generally there are two options:

1. Store in datanode (like datanodeDetails) and every time pipelines mapping change on single datanode, pipeline ids will be serialized to local file. This way will lead to many more local serialization of things like datanodeDetails, but the updates are only for local datanode change. Improvement can be made like linking a serializable object to datanodeDetails and datanode keeps updating the new pipeline ids to the serializable object instead the details file. On the other hand, since the pipeline ids are stored only in datanode locally, there will be no global view in SCM. (or we can store a lazy copy?)
2. Stored in SCM. SCM can maintain a large mapping between datanode ids and pipeline ids. But this way will lead to an exponentially increasing frequency in SCM updates since the pipeline mapping changes are way more complex and happen all the time. Obviously this gives SCM too much pressure, but it can also give SCM a global view on the management over datanodes and multi raft pipelines. 

 

Thoughts? xyao Sammi 

Issue Links

• Blocked: HDDS-1564 Ozone multi-raft support