HBase
  1. HBase
  2. HBASE-11124

Pluggable major compaction strategy

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Consider adding to the master a schedulable internal task that manages major compaction. The scheduler implementation should be pluggable. The default plugin should implement the current policy, a simple interval trigger governed by a site configuration property. Reserve a znode as a store for major compaction scheduler state, allow the plugin access to (re)create it and update as needed.

        Activity

        Andrew Purtell created issue -
        Andrew Purtell made changes -
        Field Original Value New Value
        Link This issue is part of HBASE-10856 [ HBASE-10856 ]
        Andrew Purtell made changes -
        Fix Version/s 0.98.3 [ 12326765 ]
        Hide
        Andrew Purtell added a comment -

        Also consider a policy that periodically triggers, with jitter, major compaction one region at a time according to least load.

        Show
        Andrew Purtell added a comment - Also consider a policy that periodically triggers, with jitter, major compaction one region at a time according to least load.
        Hide
        Vladimir Rodionov added a comment -

        Also consider a policy that periodically triggers, with jitter, major compaction one region at a time according to least load.

        I am wondering what are others, different policies? Doing compaction small piece at a time, taking into account region's load is the one and seems the only one we need?

        Show
        Vladimir Rodionov added a comment - Also consider a policy that periodically triggers, with jitter, major compaction one region at a time according to least load. I am wondering what are others, different policies? Doing compaction small piece at a time, taking into account region's load is the one and seems the only one we need?
        Hide
        Andrew Purtell added a comment -

        Doing compaction small piece at a time, taking into account region's load is the one and seems the only one we need?

        That and an alternative that does the backwards compatible behavior is only currently proposed here, agree.

        Show
        Andrew Purtell added a comment - Doing compaction small piece at a time, taking into account region's load is the one and seems the only one we need? That and an alternative that does the backwards compatible behavior is only currently proposed here, agree.
        Andrew Purtell made changes -
        Fix Version/s 0.98.3 [ 12326765 ]
        Enis Soztutar made changes -
        Fix Version/s 0.99.1 [ 12328551 ]
        Fix Version/s 0.99.0 [ 12325675 ]
        Enis Soztutar made changes -
        Fix Version/s 2.0.0 [ 12327188 ]
        Fix Version/s 0.99.1 [ 12328551 ]
        Hide
        Enis Soztutar added a comment -

        Unscheduling from branch-1. Feel free to bring back if there is a patch.

        Show
        Enis Soztutar added a comment - Unscheduling from branch-1. Feel free to bring back if there is a patch.
        stack made changes -
        Link This issue is part of HBASE-10856 [ HBASE-10856 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Andrew Purtell
          • Votes:
            2 Vote for this issue
            Watchers:
            16 Start watching this issue

            Dates

            • Created:
              Updated:

              Development