Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1625

Schedule compaction on rowsets with high percentage of deleted data

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.0.0
    • None
    • tablet
    • None

    Description

      Although with KUDU-236 we can now remove rows that were deleted prior to the ancient history mark, we don't actively schedule compactions based on deleted rows. So, if for example we have a fully compacted table and issue a DELETE for every row, the data size actually does not change, because no compactions are triggered.

      We need some way to notice the fact that the ratio of deletes to rows is high and decide to compact those rowsets.

      Attachments

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. KUDU-1979 Row GC maintenance task

          • Major - Major loss of function.
          • Resolved
          is related to

          New Feature - A new feature of the product, which has yet to be developed. KUDU-3367 Delta file with full of delete op can not be schedule to compact

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. KUDU-1400 Improve rowset compaction policy to consider merging small DRSs

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. KUDU-3619 The 'supplement to GC algorithm' breaks major delta compaction

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              tlipcon Todd Lipcon
              Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated: