Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-11506 Improvements for large scale deletion
  3. HDDS-11514

Set optimal default values for delete configurations based on live cluster testing

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • OM, SCM
    • None

    Description

      Some configurations to throttle background deletions have been chosen with educated guesses, and may not reflect real cluster usage. We should test large scale deletions of millions of files and directories and TBs of data all at once on a real cluster, tune the configurations accordingly, and use this experiment to inform our choice of default values.

      Attachments

        Issue Links

          requires

          Sub-task - The sub-task of the issue HDDS-11508 Decouple delete batch limits from Ratis request size

          • Major - Major loss of function.
          • Open

          Sub-task - The sub-task of the issue HDDS-11512 Create Grafana dashboard for tracking system wide deletion

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              erose Ethan Rose
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: