[HDDS-4125] Pipeline is not removed when a datanode goes stale

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: SCM HA
• Labels: pull-request-available
• Assignee: Glen Geng
• Reporter: Nandakumar
• Created: 16/Aug/20 18:31
• Updated: 21/Aug/20 06:03
• Resolved: 21/Aug/20 06:03

Description

When a node goes stale the pipelines in that node have to be closed and removed from PipelineManager. Currently, the pipeline is only closed and left in PipelineManager.

 

Root Cause Analysis 

Since the Scheduler in SCMPipelineManager that used to destroyPipeline is removed,

scheduler.schedule(() -> destroyPipeline(pipeline),
    pipelineDestroyTimeoutInMillis, TimeUnit.MILLISECONDS, LOG,
    String.format("Destroy pipeline failed for pipeline:%s", pipeline));

meanwhile the PipelineManagerV2Impl::scrubPipeline only handles and remove RATIS THREE pipeline,

public void scrubPipeline(ReplicationType type, ReplicationFactor factor)
    throws IOException {
  checkLeader();
  if (type != ReplicationType.RATIS || factor != ReplicationFactor.THREE) {
    // Only srub pipeline for RATIS THREE pipeline
    return;
  }

 RATIS ONE Pipeline is closed but not removed when a datanode goes stale. The solution is let scrubPipeline handle all kinds of pipelines.

Issue Links

• links to: GitHub Pull Request #1334