Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-5136

Recycling of pipelines in ozone

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: SCM
    • Labels:
      None

      Description

      Currently, in ozone pipelines can remain open for ever, unless there is a failure detected. The idea here is to close pipelines over a times and recycle them to achieve the below objectives:

      1) With pipelines, being active raft log disks can get full and eventually fail well before closing. As the pipelines can not be closed, the raft log disk cannot be cleaned up. Autoclosing over a time, will help keep the raft log disk space maintainable.

      2) Pipelines with datanodes with slow disk/bad disk/full disk can be closed actively and then moved out of pipeline allocation.

      3) In cases such node addition, disk additions , pipelines on nodes with lower disk capacity can be closed and new IO's can be redirected to new nodes/disks.

      4) Will help the cluster make more rack and toplogy aware over time.

      With multiraft, all the above challenges/challenges become more significant.

       

        Attachments

          Activity

            People

            • Assignee:
              shashikant Shashikant Banerjee
              Reporter:
              shashikant Shashikant Banerjee
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: