[HBASE-21355] HStore's storeSize is calculated repeatedly which causing the confusing region split

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 1.3.3, 2.2.0, 2.1.1, 2.0.3, 1.4.9
• Component/s: regionserver
• Hadoop Flags: Reviewed
• Assignee: Zheng Hu
• Reporter: Zheng Hu
• Created: 21/Oct/18 07:38
• Updated: 01/Feb/19 20:13
• Resolved: 22/Oct/18 13:36

Description

When testing the branch-2's write performance in our internal cluster, we found that the region will be inexplicably split.

We use the default ConstantSizeRegionSplitPolicy and hbase.hregion.max.filesize=40G，but the region will be split even if its bytes size is less than 40G(only ~6G).

Checked the code, I found that the following path will accumulate the store's storeSize to a very big value, because the path has no reset..

RsRpcServices#getRegionInfo
      -> HRegion#isMergeable
           -> HRegion#hasReferences
                -> HStore#hasReferences
                        -> HStore#openStoreFiles

BTW, we seems forget to maintain the read replica's storeSize when refresh the store files.

Some comment here, I move the storeSize calculation out of loadStoreFiles() method, because the secondary read replica's refreshStoreFiles() will also use loadStoreFiles() to refresh its store files and update the storeSize in the completeCompaction(..) in the final (just like compaction.) , so no need calculate the storeSize twice..

Issue Links

• is caused by: HBASE-20940 HStore.cansplit should not allow split to happen if it has references
• links to: Review board