Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21883

Enhancements to Major Compaction tool

    XMLWordPrintableJSON

    Details

    • Release Note:
      Hide
      MajorCompactorTTL Tool allows to compact all regions in a table that have been TTLed out. This saves space on DFS and is useful for tables which are similar to time series data. This is typically scheduled to run frequently (say via cron) to cleanup old data on an ongoing basis.

      RSGroupMajorCompactionTTL tool is similar to MajorCompactorTTL but runs at a region server group level. If multiple tables in an rsgroup are similar to time-series data, then it runs a single command to clean them up. As more tables are added/removed from rsgroup, it's easy to have a single command to take care of all of them.
      Show
      MajorCompactorTTL Tool allows to compact all regions in a table that have been TTLed out. This saves space on DFS and is useful for tables which are similar to time series data. This is typically scheduled to run frequently (say via cron) to cleanup old data on an ongoing basis. RSGroupMajorCompactionTTL tool is similar to MajorCompactorTTL but runs at a region server group level. If multiple tables in an rsgroup are similar to time-series data, then it runs a single command to clean them up. As more tables are added/removed from rsgroup, it's easy to have a single command to take care of all of them.

      Description

      I would like to add new compaction tools based on churro morales's tool at HBASE-19528.

      We internally have tools that pick and compact regions based on multiple criteria. Since Rahul already has a version in community, we would like to build on top of it instead of pushing yet another tool.

      With this jira, I would like to add a tool which looks at regions beyond TTL and compacts them in a rsgroup. We have time series data and those regions will become dead after a while, so we compact those regions to save disk space. We also merge those empty regions to reduce load, but that tool comes later.

      Will prep a patch for 2.x once 1.5 gets in.

        Attachments

        1. HBASE-21883.branch-1.001.patch
          60 kB
          Thiruvel Thirumoolan
        2. HBASE-21883.branch-1.002.patch
          65 kB
          Thiruvel Thirumoolan
        3. HBASE-21883.master.001.patch
          65 kB
          Thiruvel Thirumoolan
        4. HBASE-21883.master.002.patch
          66 kB
          Thiruvel Thirumoolan

          Activity

            People

            • Assignee:
              thiruvel Thiruvel Thirumoolan
              Reporter:
              thiruvel Thiruvel Thirumoolan
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: