Affects Version/s: 0.90.0, 0.90.1, 0.92.0
Fix Version/s: None
We got into an issue today where we were using HFileOutputFormat to perform an incremental load on an already-large cluster. Because bulk-loaded files don't have a sequence ID, they are put in the front of the StoreFile list. This resulted in the following StoreFile ordering
2GB (bulk) => 25GB => 2GB => ...
So this triggered a 30+GB major compaction for every single region. Optimally, we would like bulk import files to be ordered in the compaction list at the time of insertion so this can be a much smaller compaction and rely on StoreFile age for major compaction trigger.
|Transition||Time In Source Status||Execution Times||Last Executer||Last Execution Date|
|1562d 52m||1||Andrew Purtell||11/Apr/15 01:54|
|Status||Open [ 1 ]||Resolved [ 5 ]|
|Resolution||Fixed [ 1 ]|