[HDDS-1880] Decommissioning and maintenance mode in Ozone

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0
• Component/s: SCM
• Labels: pull-request-available
• Assignee: Stephen O'Donnell
• Reporter: Marton Elek
• Created: 31/Jul/19 11:55
• Updated: 21/Jan/24 15:42
• Resolved: 21/Dec/22 09:47

Description

This is the umbrella jira for decommissioning support in Ozone. Design doc will be attached soon.

Issue Links

• links to: GitHub Pull Request #1461
• links to: GitHub Pull Request #1474
• links to: GitHub Pull Request #1520
• links to: GitHub Pull Request #1568
• links to: GitHub Pull Request #1580

Sub-Tasks

• Design doc: decommissioning in Ozone: Resolved
• Extend SCMNodeManager to support decommission and maintenance states: Resolved
• Add CLI Commands and Protobuf messages to trigger decom states: Resolved
• Extend SCMCLI Topology command to print node Operational States: Resolved
• Destroy pipelines on any decommission or maintenance nodes: Resolved
• QueryNode does not respect null values for opState or state: Resolved
• ContainerReplica should contain DatanodeInfo rather than DatanodeDetails: Resolved
• Refactor ReplicationManager to consider maintenance states: Resolved
• DatanodeAdminMonitor should track under replicated containers and complete the admin workflow accordingly: Resolved
• DeadNodeHandler should not remove replica for a dead maintenance node: Resolved
• Investigate why TestDatanodeAdminMonitor.testMonitoredNodeHasPipelinesClosed() fails: Resolved
• Have NodeManager.getNodeStatus throw NodeNotFoundException: Resolved
• Remove methods of internal representation from DatanodeAdminMontor interface: Resolved
• Add Datanode command to allow the datanode to persist its admin state: Resolved
• Allow SCM webUI to show decommision and maintenance nodes: Resolved
• Consider allowing maintenance end time to be specified in human readable format: Open
• Update JMX metrics for node count in SCMNodeMetrics for Decommission and Maintenance: Resolved
• Allow users to pass hostnames or IP when decommissioning nodes: Open
• Expose decommission / maintenance metrics via JMX: Resolved
• Merge MockNodeManager and SimpleMockNodeManager: Resolved
• Cluster disk space metrics should reflect decommission and maintenance states: Resolved
• Add some unit tests around the changes in HDDS-2592: Resolved
• Consider using INFINITY in decommission and maintenance commands where not time is specified: Resolved
• Merge Master branch into decom branch: Resolved
• Investigate failure of TestDecommissionAndMaintenance integration test: Resolved
• Change replication logic to use PersistedOpState: Resolved
• Remove no longer needed class DatanodeAdminNodeDetails: Resolved
• Add integration tests for Decommission and resolve issues detected by the tests: Resolved
• Add integration tests for putting nodes into maintenance and fix any issues uncovered in the tests: Resolved
• DatanodeAdminMonitor no longers needs maintenance end time to be passed: Resolved
• Add Operational State to the datanode list command: Resolved
• Show Datanode OperationalState (IN_SERVICE/DECOMMISSION/MAINTENANCE) in Recon: Resolved
• SCM can incorrectly marks Datanode as DECOMMISSIONING when Datanode is not fully initialized: Closed
• Update NodeStatus OperationalState for Datanodes in Recon: Resolved
• Add line break when node has no pipelines for `ozone admin datanode list` command: Resolved
• Improve Ozone admin shell decommission/recommission/maintenance commands user experience: Resolved