Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-3048

unify code for major/minor compactions

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.90.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      minor compactions now enforce ttl, maxversions, deletes, etc... but will let through the delete markers themselves

      Description

      Today minor compactions do not process deletes, purge old versions, etc. Only major compactions do. The rationale was probably to save CPU . We should evaluate if major compaction logic indeed runs significantly slower.

      Unifying minor compactions to do the same thing as major compactions has other advantages:

      • If the same keys are deleted/updated repeatedly, the fact that deletes/overwrites are not processed during minor compaction makes each subsequent minor compaction more expensive as the total amount of data keeps growing.
      • We'll have fewer bugs if the logic is as symmetric as possible. Any bugs in TTL enforcement, version enforcement, etc. could cause behavior to be different after a major compaction. Keeping the same logic means these bugs will get caught earlier.

      -

      Note: There will still need to be one difference in the two schemes, and that has to do with delete markers. Any compaction which doesn't compact all files will still need to leave delete markers.

        Attachments

        1. HBASE-3048-0.90-v2.patch
          31 kB
          Jonathan Gray
        2. HBASE-3048-v3.patch
          32 kB
          Jonathan Gray
        3. unify.patch
          19 kB
          Amitanand Aiyer

          Activity

            People

            • Assignee:
              amitanand Amitanand Aiyer
              Reporter:
              kannanm Kannan Muthukkaruppan
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: