HBase
  1. HBase
  2. HBASE-5492

Caching StartKeys and EndKeys of Regions

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.92.0
    • Fix Version/s: None
    • Component/s: Client
    • Labels:
      None
    • Environment:

      all

      Description

      Each call for HTable.getStartEndKeys will read meta table.

      In particular,
      in the case of client side multi-threaded concurrency statistics,
      we must call HTable.coprocessorExec== > getStartKeysInRange ==> getStartEndKeys,
      resulting in the need to always scan the meta table.

      This is not necessary,
      we can implement the HConnectionManager.HConnectionImplementation.locateRegions(byte[] tableName) method,

      then, get the StartKeys and EndKeys from the cachedRegionLocations of HConnectionImplementation.

      Combined with https://issues.apache.org/jira/browse/HBASE-5491, can improve the performance of statistical

        Issue Links

          Activity

          Hide
          stack added a comment -

          Once filled, how does the cache of table locations get refreshed if a table region splits?

          Show
          stack added a comment - Once filled, how does the cache of table locations get refreshed if a table region splits?
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12516578/HBASE-5492.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javadoc. The javadoc tool appears to have generated -131 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 154 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.wal.TestHLog
          org.apache.hadoop.hbase.TestRegionRebalancing

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1061//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1061//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1061//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12516578/HBASE-5492.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javadoc. The javadoc tool appears to have generated -131 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 154 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.wal.TestHLog org.apache.hadoop.hbase.TestRegionRebalancing Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1061//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1061//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1061//console This message is automatically generated.
          Hide
          honghua zhu added a comment -

          @stack

          Using cachedRegionLocations of HConnectionImplementation.

          Your question can be converted into who is responsible for refreshing of cachedRegionLocations?

          Show
          honghua zhu added a comment - @stack Using cachedRegionLocations of HConnectionImplementation. Your question can be converted into who is responsible for refreshing of cachedRegionLocations?
          Hide
          stack added a comment -

          Your question can be converted into who is responsible for refreshing of cachedRegionLocations?

          You can do that. Its done as the HTable runs but there are no guarantees the cache is complete at any one time, right?

          Change + if (tableLocations.size() == 0) { to tableLocations.isEmpty().. the former can be costly compared.

          So, IIUC, you are asking to pull all region locations local in locateRegions?

          Does this get all table regions? Or just the configured next ten regions?

          + prefetchRegionCache(tableName, null);

          Show
          stack added a comment - Your question can be converted into who is responsible for refreshing of cachedRegionLocations? You can do that. Its done as the HTable runs but there are no guarantees the cache is complete at any one time, right? Change + if (tableLocations.size() == 0) { to tableLocations.isEmpty().. the former can be costly compared. So, IIUC, you are asking to pull all region locations local in locateRegions? Does this get all table regions? Or just the configured next ten regions? + prefetchRegionCache(tableName, null);
          Hide
          Gary Helmling added a comment -

          Closing as a duplicate of HBASE-6870, which has more up-to-date work on the same problem. If there's some other aspect I'm missing, feel free to re-open.

          Show
          Gary Helmling added a comment - Closing as a duplicate of HBASE-6870 , which has more up-to-date work on the same problem. If there's some other aspect I'm missing, feel free to re-open.

            People

            • Assignee:
              Unassigned
              Reporter:
              honghua zhu
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development