[HBASE-16876] RatioBasedCompactionPolicy ignores mayBeStuck

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Compaction
• Labels: newbie
• Assignee: Unassigned
• Reporter: Neal Young
• Created: 18/Oct/16 22:16
• Updated: 24/Oct/16 16:02

Description

I'm a newbie so may not be reporting this bug correctly. The bug currently shows in lines 181 - 190 here : http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.html#181 .

Bar.java

176     while (countOfFiles - start >= comConf.getMinFilesToCompact() &&
177       fileSizes[start] > Math.max(comConf.getMinCompactSize(),
178           (long) (sumSize[start + 1] * ratio))) {
179       ++start;
180     }
181     if (start < countOfFiles) {
182       LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
183         + " files from " + countOfFiles + " candidates");
184     } else if (mayBeStuck) {
185       // We may be stuck. Compact the latest files if we can.
186       int filesToLeave = candidates.size() - comConf.getMinFilesToCompact();
187       if (filesToLeave >= 0) {
188         start = filesToLeave;
189       }
190     }

On reaching line 176, start = 0. When comConf.getMinFilesToCompact() is at least 2 (as occurs in the default), the while loop is guaranteed to terminate with start < countOfFiles. Hence, the else clause starting at line 184 never executes, regardless of the value of mayBeStuck.

Perhaps the following code would be better, but I'm not sure:

Bar.java

    while (start < countOfFiles
           && fileSizes[start] > Math.max(comConf.getMinCompactSize(),
                                          (long) (sumSize[start + 1] * ratio))) {
        ++start;
    }
    if (countOfFiles - start < comConf.getMinFilesToCompact()) {
        if (mayBeStuck && countOfFiles >= comConf.getMinFilesToCompact()) {
            start = countOfFiles - comConf.getMinFilesToCompact();
        } else {
            start = countOfFiles;
        }
    }
    if (countOfFiles - start > 0) {
        LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
                 + " files from " + countOfFiles + " candidates");
    }