Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21355

HStore's storeSize is calculated repeatedly which causing the confusing region split

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      When testing the branch-2's write performance in our internal cluster, we found that the region will be inexplicably split.

      We use the default ConstantSizeRegionSplitPolicy and hbase.hregion.max.filesize=40G,but the region will be split even if its bytes size is less than 40G(only ~6G).

      Checked the code, I found that the following path will accumulate the store's storeSize to a very big value, because the path has no reset..

      RsRpcServices#getRegionInfo
            -> HRegion#isMergeable
                 -> HRegion#hasReferences
                      -> HStore#hasReferences
                              -> HStore#openStoreFiles
      

      BTW, we seems forget to maintain the read replica's storeSize when refresh the store files.

      Some comment here, I move the storeSize calculation out of loadStoreFiles() method, because the secondary read replica's refreshStoreFiles() will also use loadStoreFiles() to refresh its store files and update the storeSize in the completeCompaction(..) in the final (just like compaction.) , so no need calculate the storeSize twice..

        Attachments

        1. HBASE-21355.v1.patch
          6 kB
          Zheng Hu
        2. HBASE-21355.branch-1.patch
          6 kB
          Zheng Hu
        3. HBASE-21355.addendum.patch
          2 kB
          Zheng Hu
        4. HBASE-21355.addendum.patch
          3 kB
          Zheng Hu

          Issue Links

            Activity

              People

              • Assignee:
                openinx Zheng Hu
                Reporter:
                openinx Zheng Hu
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: