[HBASE-26249] Ameliorate compaction made by bulk-loading files

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.5.0, 3.0.0-alpha-2
• Assignee: Xiaolin Ha
• Reporter: Xiaolin Ha
• Created: 02/Sep/21 06:45
• Updated: 23/Jun/22 05:06
• Resolved: 23/Nov/21 08:01

Description

HBASE-25213 introduced to trigger compact after bulkload, but if a store needs compaction, each file loaded to this store will add a compaction runner of this store to the compaction queue. Then repeatedly compute for needCompaction and selectFiles for this store, and the compaction of the store may be continuous.

We can mark the queued compaction runners for the stores, and avoid repeatedly adding same store compactions to the queue after bulk load files. This method can reduce the pressure during the period just after bulk loading and spread the pressure to other times by the periodic CompactionChecker.

For the need of splitting mentioned before, after the completion of compaction, it will trigger split of the region if it needs. So we do not need to trigger split after bulk loading.

 

Issue Links

• causes: HBASE-27152 Under compaction mark may leak
• links to: GitHub Pull Request #3658
• links to: GitHub Pull Request #3831
• links to: GitHub Pull Request #3874