[HDDS-6884] Allow writes while finalizing non-invasive layout features.

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Ozone Datanode, SCM
• Assignee: Ethan Rose
• Reporter: Ethan Rose
• Created: 14/Jun/22 19:56
• Updated: 23/Mar/23 18:33

Description

Currently SCM closes all containers and pipelines during finalization, so that a datanode must finalize before it writes new data. For more invasive layout features this may be necessary, but in the common case it generates container churn and slows down the finalization process. The majority of layout features seen so far involve a simple on/off switch and minor additions to the disk state, so we should provide a way to bypass the "close everything" step if it is not needed.

This jira proposes allowing unfinalized datanodes to continue serving writes if the layout features needing to be finalized are simple. In this case SCM finalization could skip closing containers and pipelines.

cc kerneltime who initially proposed this improvement to the finalization process.

Issue Links

• is a child of: HDDS-5444 Ozone Upgrades vNext
• is related to: HDDS-8264 SCM HA Finalization v2

Sub-Tasks

• Allow HDDS layout features to specify their finalization requirements: Patch Available
• Make SCM determine the schema version of new containers: Resolved
• No longer use HEALTHY_READONLY state to indicate pre-finalized datanodes: Open
• Manage layout version of pipelines: Open