HBase
  1. HBase
  2. HBASE-10058

Test for HBASE-9915 (avoid reading index blocks)

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.94.15, 0.96.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In HBASE-9915 we found that for encoded blocks the quick check whether the seek key falls on the current block did not work.

      This adds a unittest so we won't regress. The test counts the number of blocks accessed (real data/index block in 0.94 and cache hits in trunk). Without HBASE-9915 we see about 50% more blocks hit in this test.

      1. 10058-0.94.txt
        4 kB
        Lars Hofhansl
      2. 10058-trunk.txt
        5 kB
        Lars Hofhansl

        Activity

        Hide
        Lars Hofhansl added a comment -

        Patch for 0.94.

        Show
        Lars Hofhansl added a comment - Patch for 0.94.
        Hide
        Lars Hofhansl added a comment -

        In 0.94 the new test fails without HBASE-9915 (touches more blocks than necessary) and passes with HBASE-9915 in place. Had we had a test like this in the beginning this HBASE-9915 would not have gone unnoticed for a year.

        I have no idea what TestBlocksScanned in trunk is supposed to test, but it sure as hell does not test "blocks scanned". It doesn't count any block metrics in trunk (maybe that got removed during the metric switch).

        I'll make this 0.94 only for now.

        Simple test-only change, I assume there are no objections.

        Show
        Lars Hofhansl added a comment - In 0.94 the new test fails without HBASE-9915 (touches more blocks than necessary) and passes with HBASE-9915 in place. Had we had a test like this in the beginning this HBASE-9915 would not have gone unnoticed for a year. I have no idea what TestBlocksScanned in trunk is supposed to test, but it sure as hell does not test "blocks scanned". It doesn't count any block metrics in trunk (maybe that got removed during the metric switch). I'll make this 0.94 only for now. Simple test-only change, I assume there are no objections.
        Hide
        Lars Hofhansl added a comment -

        Test for trunk. I reinstated the block counts (by using CacheStats).
        In general we should be careful to not cripple tests.

        If no objections I will commit later today.

        Show
        Lars Hofhansl added a comment - Test for trunk. I reinstated the block counts (by using CacheStats). In general we should be careful to not cripple tests. If no objections I will commit later today.
        Hide
        stack added a comment -

        lgtm

        Show
        stack added a comment - lgtm
        Hide
        Lars Hofhansl added a comment -

        Committed to all branches. Thanks Stack.

        Show
        Lars Hofhansl added a comment - Committed to all branches. Thanks Stack.
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.94-security #349 (See https://builds.apache.org/job/HBase-0.94-security/349/)
        HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546659)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.94-security #349 (See https://builds.apache.org/job/HBase-0.94-security/349/ ) HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546659) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-0.94 #1215 (See https://builds.apache.org/job/HBase-0.94/1215/)
        HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546659)

        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-0.94 #1215 (See https://builds.apache.org/job/HBase-0.94/1215/ ) HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546659) /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK #4703 (See https://builds.apache.org/job/HBase-TRUNK/4703/)
        HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546658)

        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK #4703 (See https://builds.apache.org/job/HBase-TRUNK/4703/ ) HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546658) /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in hbase-0.96-hadoop2 #137 (See https://builds.apache.org/job/hbase-0.96-hadoop2/137/)
        HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546660)

        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Show
        Hudson added a comment - SUCCESS: Integrated in hbase-0.96-hadoop2 #137 (See https://builds.apache.org/job/hbase-0.96-hadoop2/137/ ) HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546660) /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #857 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/857/)
        HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546658)

        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #857 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/857/ ) HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546658) /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in hbase-0.96 #208 (See https://builds.apache.org/job/hbase-0.96/208/)
        HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546660)

        • /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java
        Show
        Hudson added a comment - SUCCESS: Integrated in hbase-0.96 #208 (See https://builds.apache.org/job/hbase-0.96/208/ ) HBASE-10058 Test for HBASE-9915 (avoid reading index blocks) (larsh: rev 1546660) /hbase/branches/0.96/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestBlocksScanned.java

          People

          • Assignee:
            Lars Hofhansl
            Reporter:
            Lars Hofhansl
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development