[HBASE-17674] Major compaction may be cancelled in CompactionChecker

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0, 1.2.4, 0.98.24, 2.0.0
• Fix Version/s: 1.4.0, 2.0.0
• Component/s: Compaction
• Hadoop Flags: Reviewed
• Assignee: Guangxu Cheng
• Reporter: Guangxu Cheng
• Created: 21/Feb/17 08:29
• Updated: 03/May/17 17:51
• Resolved: 26/Feb/17 17:38

Description

CompactionChecker will periodically check whether a region should perform a major compaction.
If a region should perform a major compact, a request is submitted. But before the request is submitted, the variable forceMajor is not set to true by calling triggerMajorCompaction.
When filtering a storefile, the large storefile may cause the request to be canceled or became minor compact.

HRegionServer.java

@Override
    protected void chore() {
      for (Region r : this.instance.onlineRegions.values()) {
        if (r == null)
          continue;
        for (Store s : r.getStores()) {
          try {
            long multiplier = s.getCompactionCheckMultiplier();
            assert multiplier > 0;
            if (iteration % multiplier != 0) continue;
            if (s.needsCompaction()) {
              // Queue a compaction. Will recognize if major is needed.
              this.instance.compactSplitThread.requestSystemCompaction(r, s, getName()
                  + " requests compaction");
            } else if (s.isMajorCompaction()) {              
              if (majorCompactPriority == DEFAULT_PRIORITY
                  || majorCompactPriority > ((HRegion)r).getCompactPriority()) {
                this.instance.compactSplitThread.requestCompaction(r, s, getName()
                    + " requests major compaction; use default priority", null);
              } else {
                this.instance.compactSplitThread.requestCompaction(r, s, getName()
                    + " requests major compaction; use configured priority",
                  this.majorCompactPriority, null, null);
              }
            }
          } catch (IOException e) {
            LOG.warn("Failed major compaction check on " + r, e);
          }
        }
      }
      iteration = (iteration == Long.MAX_VALUE) ? 0 : (iteration + 1);
    }