Major Description
The following came up as part of the review and related discussions with avijayan about HDDS-4219 but as these things came in earlier, we decided to open up a new JIRA for this, to discuss what can be done. CC ppogde as he worked the most on implementing the HDDS part.
UpgradeAction is part of the LayoutFeature interface, but its descendants HDDSUpgradeAction and OMUpgradeAction are separate interfaces, without a specific LayoutFeature. This seems to be a flaw, especially because concrete implementations of the UpgradeAction interface are not doing any component specific thing that is not covered by the base interface, and probably they won’t do.
The way I look at it the UpgradeAction at the moment is, and may remain in the future also, a way to support a callback mechanism during the upgrade for the component to add necessary steps in order to activate a new feature. If this is true, probably we can even use the Consumer interface instead (more on this idea later), or at least mark the interface to be a functional interface, and use it that way.
The relationship between LayoutFeatures and UpgradeActions are having two kind of binding in OM and in HDDS, I think the OM one is the better approach.
In OM, we have OMLayoutFeature enum, which implements the LayoutFeature interface, and which defines the LayoutFeatures with layoutVersion, description, and UpgradeAction triplets, while in HDDS, we have DataNodeLayoutAction, and SCMLayoutAction enums that binds the UpgradeActions to LayoutFeatures independently from HDDSLayoutFeature, and therefore we have HDDSLayoutFeature as an enum, as it is by nature, but with mutable fields, which we then carefully mutate at initialization time.
The biggest problem with this is that, HDDSLayoutFeature basically redefines the LayoutFeature API, adds onFinalizeSCMAction, and onFinalizeDataNodeAction methods to the interface, and let the rest of the code neglect the onFinalizeAction part of the HDDSLayoutFeatures.
The rationale behind this is probably the fact that HDDS has two components, and a LayoutFeature may belong to only one of them, or both of them, and in case of both, the LayoutFeature API is not able to provide a type consistent view of the UpgradeActions, as there can be UpgradeAction<DatanodeStateMachine> and UpgradeAction<StorageContainerManager> actions associated with a LayoutFeature.
The control flow that is using this part of the LayoutFeature is finalization. We have three very similar way of finalizing the 3 component which we support at the moment. But all of them has subtle differences, some may be accidental, and some comes from different APIs, but I firmly believe this can be a generic flow of the following steps:
- general prepare of finalization if any
- loop over the actions
- call the finalization callback of actions if any
- report the results back
- finish finalization do some after the fact things if any
- report ready to the caller
I am unsure yet what has to be done for this generalization, but I believe there is an approach where these problems are not present and the control flow remains the same with more generic and with that shorter and standardized solutions.
I am happy to put work into this but I don't have time until the next week for sure.