HBase
  1. HBase
  2. HBASE-5274

Filter out the expired store file scanner during the compaction

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files.

      This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files.

      1. ASF.LICENSE.NOT.GRANTED--D1407.1.patch
        4 kB
        Phabricator
      2. ASF.LICENSE.NOT.GRANTED--D1407.1.patch
        4 kB
        Phabricator
      3. ASF.LICENSE.NOT.GRANTED--D1407.1.patch
        4 kB
        Phabricator
      4. ASF.LICENSE.NOT.GRANTED--D1407.1.patch
        4 kB
        Phabricator
      5. ASF.LICENSE.NOT.GRANTED--D1407.1.patch
        4 kB
        Phabricator
      6. ASF.LICENSE.NOT.GRANTED--D1473.1.patch
        12 kB
        Phabricator

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #2648 (See https://builds.apache.org/job/HBase-TRUNK/2648/)
          [jira] HBASE-5274 Filter out expired scanners on compaction as well

          Summary: This is a followup for D1017 to make it similar to D909 (89-fb). The
          fix for 89-fb used the TTL-based scanner filtering logic on both normal scanners
          and compactions, while the trunk fix D1017 did not. This is just the delta
          between the two diffs that brings filtering expired store files on compaction to
          trunk.

          Test Plan: Unit tests

          Reviewers: Liyin, JIRA, lhofhansl, Kannan

          Reviewed By: Liyin

          CC: Liyin, tedyu, Kannan, mbautin, lhofhansl

          Differential Revision: https://reviews.facebook.net/D1473

          mbautin :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #2648 (See https://builds.apache.org/job/HBase-TRUNK/2648/ ) [jira] HBASE-5274 Filter out expired scanners on compaction as well Summary: This is a followup for D1017 to make it similar to D909 (89-fb). The fix for 89-fb used the TTL-based scanner filtering logic on both normal scanners and compactions, while the trunk fix D1017 did not. This is just the delta between the two diffs that brings filtering expired store files on compaction to trunk. Test Plan: Unit tests Reviewers: Liyin, JIRA, lhofhansl, Kannan Reviewed By: Liyin CC: Liyin, tedyu, Kannan, mbautin, lhofhansl Differential Revision: https://reviews.facebook.net/D1473 mbautin : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-security #92 (See https://builds.apache.org/job/HBase-TRUNK-security/92/)
          [jira] HBASE-5274 Filter out expired scanners on compaction as well

          Summary: This is a followup for D1017 to make it similar to D909 (89-fb). The
          fix for 89-fb used the TTL-based scanner filtering logic on both normal scanners
          and compactions, while the trunk fix D1017 did not. This is just the delta
          between the two diffs that brings filtering expired store files on compaction to
          trunk.

          Test Plan: Unit tests

          Reviewers: Liyin, JIRA, lhofhansl, Kannan

          Reviewed By: Liyin

          CC: Liyin, tedyu, Kannan, mbautin, lhofhansl

          Differential Revision: https://reviews.facebook.net/D1473

          mbautin :
          Files :

          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
          • /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java
          • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-security #92 (See https://builds.apache.org/job/HBase-TRUNK-security/92/ ) [jira] HBASE-5274 Filter out expired scanners on compaction as well Summary: This is a followup for D1017 to make it similar to D909 (89-fb). The fix for 89-fb used the TTL-based scanner filtering logic on both normal scanners and compactions, while the trunk fix D1017 did not. This is just the delta between the two diffs that brings filtering expired store files on compaction to trunk. Test Plan: Unit tests Reviewers: Liyin, JIRA, lhofhansl, Kannan Reviewed By: Liyin CC: Liyin, tedyu, Kannan, mbautin, lhofhansl Differential Revision: https://reviews.facebook.net/D1473 mbautin : Files : /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
          Hide
          Mikhail Bautin added a comment -

          Fix committed to trunk.

          Show
          Mikhail Bautin added a comment - Fix committed to trunk.
          Hide
          Phabricator added a comment -

          mbautin has committed the revision "[jira] HBASE-5274 Filter out expired scanners on compaction as well".

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          COMMIT
          https://reviews.facebook.net/rHBASE1236483

          Show
          Phabricator added a comment - mbautin has committed the revision " [jira] HBASE-5274 Filter out expired scanners on compaction as well". REVISION DETAIL https://reviews.facebook.net/D1473 COMMIT https://reviews.facebook.net/rHBASE1236483
          Hide
          Phabricator added a comment -

          lhofhansl has commented on the revision "[jira] HBASE-5274 Filter out expired scanners on compaction as well".

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:975 Fair enough. +1

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          Show
          Phabricator added a comment - lhofhansl has commented on the revision " [jira] HBASE-5274 Filter out expired scanners on compaction as well". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:975 Fair enough. +1 REVISION DETAIL https://reviews.facebook.net/D1473
          Hide
          Phabricator added a comment -

          mbautin has commented on the revision "[jira] HBASE-5274 Filter out expired scanners on compaction as well".

          Lars: please see my response inline.

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:975 In that case I would have to make LruBlockCache.getCachedFileNamesForTest public. In addition, this patch makes HBASE-5010 implementation consistent in 89-fb and trunk, and moving the unit test around might create confusion.

          Please let me know if this is OK to commit.

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          Show
          Phabricator added a comment - mbautin has commented on the revision " [jira] HBASE-5274 Filter out expired scanners on compaction as well". Lars: please see my response inline. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:975 In that case I would have to make LruBlockCache.getCachedFileNamesForTest public. In addition, this patch makes HBASE-5010 implementation consistent in 89-fb and trunk, and moving the unit test around might create confusion. Please let me know if this is OK to commit. REVISION DETAIL https://reviews.facebook.net/D1473
          Hide
          Phabricator added a comment -

          lhofhansl has commented on the revision "[jira] HBASE-5274 Filter out expired scanners on compaction as well".

          +1

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:975 Minor comment: Is there no way to move the tests into the same package and leave this protected?

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          Show
          Phabricator added a comment - lhofhansl has commented on the revision " [jira] HBASE-5274 Filter out expired scanners on compaction as well". +1 INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/Store.java:975 Minor comment: Is there no way to move the tests into the same package and leave this protected? REVISION DETAIL https://reviews.facebook.net/D1473
          Hide
          Phabricator added a comment -

          tedyu has commented on the revision "[jira] HBASE-5274 Filter out expired scanners on compaction as well".

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:218 That should be fine.
          Another approach is to use comment directly.

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          Show
          Phabricator added a comment - tedyu has commented on the revision " [jira] HBASE-5274 Filter out expired scanners on compaction as well". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:218 That should be fine. Another approach is to use comment directly. REVISION DETAIL https://reviews.facebook.net/D1473
          Hide
          Phabricator added a comment -

          mbautin has commented on the revision "[jira] HBASE-5274 Filter out expired scanners on compaction as well".

          Ted: replying to your comment inline. Please let me know if this is OK to be committed.

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:218 @tedyu: this is just for clarity. Boolean parameters are inherently confusing, and this is an equivalent of a comment saying that "false" means "isCompaction".

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          Show
          Phabricator added a comment - mbautin has commented on the revision " [jira] HBASE-5274 Filter out expired scanners on compaction as well". Ted: replying to your comment inline. Please let me know if this is OK to be committed. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:218 @tedyu: this is just for clarity. Boolean parameters are inherently confusing, and this is an equivalent of a comment saying that "false" means "isCompaction". REVISION DETAIL https://reviews.facebook.net/D1473
          Hide
          Phabricator added a comment -

          Kannan has commented on the revision "[jira] HBASE-5274 Filter out expired scanners on compaction as well".

          +1

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          Show
          Phabricator added a comment - Kannan has commented on the revision " [jira] HBASE-5274 Filter out expired scanners on compaction as well". +1 REVISION DETAIL https://reviews.facebook.net/D1473
          Hide
          Phabricator added a comment -

          tedyu has commented on the revision "[jira] HBASE-5274 Filter out expired scanners on compaction as well".

          Looks good.

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:218 isCompaction is not needed, can pass false directly.

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          Show
          Phabricator added a comment - tedyu has commented on the revision " [jira] HBASE-5274 Filter out expired scanners on compaction as well". Looks good. INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:218 isCompaction is not needed, can pass false directly. REVISION DETAIL https://reviews.facebook.net/D1473
          Hide
          Phabricator added a comment -

          Liyin has accepted the revision "[jira] HBASE-5274 Filter out expired scanners on compaction as well".

          LGTM. Thanks Mikhail !

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          Show
          Phabricator added a comment - Liyin has accepted the revision " [jira] HBASE-5274 Filter out expired scanners on compaction as well". LGTM. Thanks Mikhail ! REVISION DETAIL https://reviews.facebook.net/D1473
          Hide
          Phabricator added a comment -

          mbautin requested code review of "[jira] HBASE-5274 Filter out expired scanners on compaction as well".
          Reviewers: Liyin, JIRA, lhofhansl, Kannan

          This is a followup for D1017 to make it similar to D909 (89-fb). The fix for 89-fb used the TTL-based scanner filtering logic on both normal scanners and compactions, while the trunk fix D1017 did not. This is just the delta between the two diffs that brings filtering expired store files on compaction to trunk.

          TEST PLAN
          Unit tests

          REVISION DETAIL
          https://reviews.facebook.net/D1473

          AFFECTED FILES
          src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
          src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java
          src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java
          src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/3063/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - mbautin requested code review of " [jira] HBASE-5274 Filter out expired scanners on compaction as well". Reviewers: Liyin, JIRA, lhofhansl, Kannan This is a followup for D1017 to make it similar to D909 (89-fb). The fix for 89-fb used the TTL-based scanner filtering logic on both normal scanners and compactions, while the trunk fix D1017 did not. This is just the delta between the two diffs that brings filtering expired store files on compaction to trunk. TEST PLAN Unit tests REVISION DETAIL https://reviews.facebook.net/D1473 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/Store.java src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java src/main/java/org/apache/hadoop/hbase/regionserver/metrics/SchemaMetrics.java src/test/java/org/apache/hadoop/hbase/io/hfile/TestScannerSelectionUsingTTL.java src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/3063/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Mikhail Bautin added a comment -

          This JIRA is a follow-up for HBASE-5010 Filter HFiles based on TTL.

          Show
          Mikhail Bautin added a comment - This JIRA is a follow-up for HBASE-5010 Filter HFiles based on TTL.
          Hide
          Phabricator added a comment -

          Liyin has abandoned the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue.

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - Liyin has abandoned the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue. REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Phabricator added a comment -

          Liyin has abandoned the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue.

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - Liyin has abandoned the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue. REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Phabricator added a comment -

          Liyin has abandoned the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue.

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - Liyin has abandoned the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue. REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Phabricator added a comment -

          Liyin has abandoned the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue.

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - Liyin has abandoned the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue. REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Phabricator added a comment -

          Liyin has abandoned the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue.

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - Liyin has abandoned the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". This could be part of HBASE-5010 Filter HFiles based on TTL. Mikhail will help to followup for fixing this issue. REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Phabricator added a comment -

          tedyu has commented on the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please.
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ?

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - tedyu has commented on the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please. src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ? REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Phabricator added a comment -

          tedyu has commented on the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please.
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ?

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - tedyu has commented on the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please. src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ? REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Phabricator added a comment -

          tedyu has commented on the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please.
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ?

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - tedyu has commented on the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please. src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ? REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Phabricator added a comment -

          tedyu has commented on the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please.
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ?

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - tedyu has commented on the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please. src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ? REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Phabricator added a comment -

          tedyu has commented on the revision "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".

          INLINE COMMENTS
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please.
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ?

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          Show
          Phabricator added a comment - tedyu has commented on the revision " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". INLINE COMMENTS src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:71 javadoc for the new fields, please. src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java:224 Can we have a more intuitive method name ? REVISION DETAIL https://reviews.facebook.net/D1407
          Hide
          Liyin Tang added a comment -

          I am not sure why the Phabricator add so many duplicated comments. Sorry about the spamming.
          @Todd, HBase-5274 tries to avoid scanning any data from the expired store file scanner. So compacting the expired store file will be very cheap. And HBase-5199 actually is related to HBASE-4717. which performs the "age-out" compaction.

          Show
          Liyin Tang added a comment - I am not sure why the Phabricator add so many duplicated comments. Sorry about the spamming. @Todd, HBase-5274 tries to avoid scanning any data from the expired store file scanner. So compacting the expired store file will be very cheap. And HBase-5199 actually is related to HBASE-4717 . which performs the "age-out" compaction.
          Hide
          Phabricator added a comment -

          Liyin requested code review of "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".
          Reviewers: Kannan, Karthik, khemani, mbautin

          During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files.

          This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files.

          TEST PLAN
          running all the unit tests

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          AFFECTED FILES
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/2931/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - Liyin requested code review of " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". Reviewers: Kannan, Karthik, khemani, mbautin During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files. This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files. TEST PLAN running all the unit tests REVISION DETAIL https://reviews.facebook.net/D1407 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/2931/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Phabricator added a comment -

          Liyin requested code review of "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".
          Reviewers: Kannan, Karthik, khemani, mbautin

          During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files.

          This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files.

          TEST PLAN
          running all the unit tests

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          AFFECTED FILES
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/2931/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - Liyin requested code review of " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". Reviewers: Kannan, Karthik, khemani, mbautin During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files. This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files. TEST PLAN running all the unit tests REVISION DETAIL https://reviews.facebook.net/D1407 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/2931/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Phabricator added a comment -

          Liyin requested code review of "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".
          Reviewers: Kannan, Karthik, khemani, mbautin

          During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files.

          This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files.

          TEST PLAN
          running all the unit tests

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          AFFECTED FILES
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/2931/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - Liyin requested code review of " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". Reviewers: Kannan, Karthik, khemani, mbautin During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files. This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files. TEST PLAN running all the unit tests REVISION DETAIL https://reviews.facebook.net/D1407 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/2931/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Phabricator added a comment -

          Liyin requested code review of "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".
          Reviewers: Kannan, Karthik, khemani, mbautin

          During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files.

          This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files.

          TEST PLAN
          running all the unit tests

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          AFFECTED FILES
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/2931/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - Liyin requested code review of " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". Reviewers: Kannan, Karthik, khemani, mbautin During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files. This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files. TEST PLAN running all the unit tests REVISION DETAIL https://reviews.facebook.net/D1407 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/2931/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Phabricator added a comment -

          Liyin requested code review of "[jira]HBASE-5274 Filter out the expired store file scanner during the compaction".
          Reviewers: Kannan, Karthik, khemani, mbautin

          During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files.

          This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files.

          TEST PLAN
          running all the unit tests

          REVISION DETAIL
          https://reviews.facebook.net/D1407

          AFFECTED FILES
          src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java

          MANAGE HERALD DIFFERENTIAL RULES
          https://reviews.facebook.net/herald/view/differential/

          WHY DID I GET THIS EMAIL?
          https://reviews.facebook.net/herald/transcript/2931/

          Tip: use the X-Herald-Rules header to filter Herald messages in your client.

          Show
          Phabricator added a comment - Liyin requested code review of " [jira] HBASE-5274 Filter out the expired store file scanner during the compaction". Reviewers: Kannan, Karthik, khemani, mbautin During the compaction time, HBase will generate a store scanner which will scan a list of store files. And it would be more efficient to filer out the expired store file since there is no need to read any key values from these store files. This optimization has been already implemented on 89-fb and this is the building block for HBASE-5199 as well. It is supposed to be no-ops to compact the expired store files. TEST PLAN running all the unit tests REVISION DETAIL https://reviews.facebook.net/D1407 AFFECTED FILES src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/2931/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
          Hide
          Todd Lipcon added a comment -

          Hey Liyin. Can you take a look at the use case in HBASE-4717? It looks related - maybe it or one of these can be closed as dup.

          Show
          Todd Lipcon added a comment - Hey Liyin. Can you take a look at the use case in HBASE-4717 ? It looks related - maybe it or one of these can be closed as dup.

            People

            • Assignee:
              Mikhail Bautin
              Reporter:
              Liyin Tang
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development