Uploaded image for project: 'Hadoop Distributed Data Store'
  1. Hadoop Distributed Data Store
  2. HDDS-3915

Simple trash emptier on OM

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Abandoned
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Ozone Manager

      Description

      Implementation something similar to HDFS's NameNode#startTrashEmptier in OzoneManager to enable automatic trash clean up.

      Some thoughts:
      1. Ozone doesn't support efficient directory renaming (not until HDDS-2939 is fully merged).

      • Possible solution: Override TrashPolicyDefault by setting fs.trash.classname in core-site.xml. So we can move files under /.Trash/<timestamp_of_delete>/ instead of /.Trash/Current/ to avoid folder renaming during checkpointing.
      • But this fs.trash.classname might affect ALL other FileSystems if configured in core-site.xml.
      • If there a way to only apply the config to o3fs and ofs.

      Update:

      In the design doc, #4 is the one we are looking at. But that approach is blocked by HDDS-3620 if we need an elegant way to implement batch rename on server side.

        Attachments

        1. Trash Cleanup Designs v1.pdf
          62 kB
          Siyao Meng

          Issue Links

            Activity

              People

              • Assignee:
                smeng Siyao Meng
                Reporter:
                smeng Siyao Meng
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: