Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1400

Improve rowset compaction policy to consider merging small DRSs

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 1.9.0
    • Component/s: None
    • Labels:
      None

      Description

      We see some small table with light write load generate lot's of small DRS(~1MB), since those DRSes do not overlap much, they don't get the chance to be compacted, generating lot of very small files/blocks. So:

      1. Compaction solution value should consider benefits of merging small DRS
      2. Every 2 min flushing MRS(small or large) seems suboptimal, maybe flushing small MRS should have "lower priority" than rowset compaction with higher solution value?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                wdberkeley William Berkeley
                Reporter:
                decster Binglin Chang
              • Votes:
                1 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: