[HBASE-14263] ExploringCompactionPolicy logic around file selection is broken

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Assignee: Unassigned
• Reporter: Vladimir Rodionov
• Created: 20/Aug/15 00:00
• Updated: 01/Jul/22 20:59
• Resolved: 27/Aug/15 20:48

Description

It seems that logic around selection of store file candidates is broken:

        // Compute the total size of files that will
        // have to be read if this set of files is compacted.
        long size = getTotalStoreSize(potentialMatchFiles);
        // Store the smallest set of files.  This stored set of files will be used
        // if it looks like the algorithm is stuck.
        if (mightBeStuck && size < smallestSize) {
          smallest = potentialMatchFiles;
          smallestSize = size;
        }
        if (size > comConf.getMaxCompactSize()) {
          continue;
        }

        ++opts;
        if (size >= comConf.getMinCompactSize()
            && !filesInRatio(potentialMatchFiles, currentRatio)) {
          continue;
        }

This is from applyCompactionPolicy method. As you can see, both min compaction size and max compaction size are applied to a selection of files and not to individual files. It mostly works as expected only because nobody seems using non-default hbase.hstore.compaction.max.size, which is Long.MAX_VALUE and it is not that easy to figure out what is going on on an opposite side (why small files do not get included?)

Issue Links

• relates to: HBASE-14326 HBase book: fix definition of max min size to compact