Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-2824

Make some tables in high priority in MM compaction

    XMLWordPrintableJSON

    Details

      Description

      In a Kudu cluster with thousands of tables, it's hard for a specified tablet's maintenance OPs to be launched when their scores are not the highest, even if the table the tablet belongs to is high priority for Kudu users.

      For example, table A has 10 tablets and has total size of 1G, table B has 1000 tablets and has total size of 100G. Both of them have similar update writes, i.e. DRSs have similar overlaps, similar redo/undo logs, so they have similar compaction scores. However, table A has much more reads than table B, but table A and B are equal in MM, their DRS compactions are lauched equally, we have to suffer a long time util most of tablets have been compacted in the cluster to achieve a fast scan.

      So, maybe we can introduce some algorithm to detect high priority tables and speed up compaction of these tables?

        Attachments

          Activity

            People

            • Assignee:
              acelyc111 Yingchun Lai
              Reporter:
              acelyc111 Yingchun Lai
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: