Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0
    • Component/s: Compaction
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Initial documentation for stripe compactions (distill from attached docs, make up to date, put somewhere like book)

      1. HBASE-9854.patch
        8 kB
        Sergey Shelukhin

        Activity

        Hide
        lars_francke Lars Francke added a comment -

        This issue was closed as part of a bulk closing operation on 2015-11-20. All issues that have been resolved and where all fixVersions have been released have been closed (following discussions on the mailing list).

        Show
        lars_francke Lars Francke added a comment - This issue was closed as part of a bulk closing operation on 2015-11-20. All issues that have been resolved and where all fixVersions have been released have been closed (following discussions on the mailing list).
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #852 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/852/)
        HBASE-9854 initial documentation for stripe compactions (stack: rev 1545382)

        • /hbase/trunk/src/main/docbkx/book.xml
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #852 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/852/ ) HBASE-9854 initial documentation for stripe compactions (stack: rev 1545382) /hbase/trunk/src/main/docbkx/book.xml
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #4698 (See https://builds.apache.org/job/HBase-TRUNK/4698/)
        HBASE-9854 initial documentation for stripe compactions (stack: rev 1545382)

        • /hbase/trunk/src/main/docbkx/book.xml
        Show
        hudson Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #4698 (See https://builds.apache.org/job/HBase-TRUNK/4698/ ) HBASE-9854 initial documentation for stripe compactions (stack: rev 1545382) /hbase/trunk/src/main/docbkx/book.xml
        Hide
        sershe Sergey Shelukhin added a comment -

        Thanks!

        Show
        sershe Sergey Shelukhin added a comment - Thanks!
        Hide
        stack stack added a comment -

        Committed for Sergey Shelukhin

        Show
        stack stack added a comment - Committed for Sergey Shelukhin
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12613960/HBASE-9854.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 hadoop1.0. The patch compiles against the hadoop 1.0 profile.

        +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        -1 site. The patch appears to cause mvn site goal to fail.

        +1 core tests. The patch passed unit tests in .

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12613960/HBASE-9854.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop1.0 . The patch compiles against the hadoop 1.0 profile. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 -1 site . The patch appears to cause mvn site goal to fail. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7868//console This message is automatically generated.
        Hide
        sershe Sergey Shelukhin added a comment -

        Patch. I am not able to build mvn site, fails with OOM seemingly no matter what I do. But, it did catch syntax problems when there were some before going to OOM, so I assume it's mostly validated.

        Show
        sershe Sergey Shelukhin added a comment - Patch. I am not able to build mvn site, fails with OOM seemingly no matter what I do. But, it did catch syntax problems when there were some before going to OOM, so I assume it's mostly validated.
        Hide
        ndimiduk Nick Dimiduk added a comment -
        Show
        ndimiduk Nick Dimiduk added a comment - How about http://hbase.apache.org/book.html#compaction ?
        Hide
        sershe Sergey Shelukhin added a comment -

        Hmm... which part of doc should this go to?

        Show
        sershe Sergey Shelukhin added a comment - Hmm... which part of doc should this go to?
        Hide
        sershe Sergey Shelukhin added a comment -

        I will make a doc patch, hopefully tomorrow or day after.
        You do want to use it for uniform tables if your regions are large, at the cost of slightly more I/O (w/flush-into-stripes) you can get more stable behavior. In my tests on multi-Gb regions reads were also slightly faster with 10-12 stripes, although I didn't investigate why then

        Show
        sershe Sergey Shelukhin added a comment - I will make a doc patch, hopefully tomorrow or day after. You do want to use it for uniform tables if your regions are large, at the cost of slightly more I/O (w/flush-into-stripes) you can get more stable behavior. In my tests on multi-Gb regions reads were also slightly faster with 10-12 stripes, although I didn't investigate why then
        Hide
        stack stack added a comment -

        Sergey Shelukhin Very nice doc especially the way you introduce the feature. Want me to add it into the doc for you?

        To set it back to default, should you change the blocking files config back down? (YOu don't mention it).

        By default, the flush creates several files from one memstore, according to existing stripe boundaries and row keys to flush.

        Is this true? By default we will flush direct to stripes? If so, that is great.

        For contrast, make mention of what M. C. Srivas noted a while back that you do NOT want to enable this feature if you are filling tables evenly because it will make for more i/o.

        Show
        stack stack added a comment - Sergey Shelukhin Very nice doc especially the way you introduce the feature. Want me to add it into the doc for you? To set it back to default, should you change the blocking files config back down? (YOu don't mention it). By default, the flush creates several files from one memstore, according to existing stripe boundaries and row keys to flush. Is this true? By default we will flush direct to stripes? If so, that is great. For contrast, make mention of what M. C. Srivas noted a while back that you do NOT want to enable this feature if you are filling tables evenly because it will make for more i/o.
        Hide
        yuzhihong@gmail.com Ted Yu added a comment -

        Nice.

        sizeToSplit = splitPartsCount

        Typo above - should be splitPartCount

        Show
        yuzhihong@gmail.com Ted Yu added a comment - Nice. sizeToSplit = splitPartsCount Typo above - should be splitPartCount
        Hide
        enis Enis Soztutar added a comment -

        This looks excellent. +1.

        Show
        enis Enis Soztutar added a comment - This looks excellent. +1.
        Hide
        sershe Sergey Shelukhin added a comment -

        stack what do you think? above comment

        Show
        sershe Sergey Shelukhin added a comment - stack what do you think? above comment
        Hide
        sershe Sergey Shelukhin added a comment -

        Here's a preview... wdyt?

        Introduction

        Stripe compactions is an experimental feature added in HBase 0.98 which aims to improve compactions for large regions or non-uniformly distributed row keys. In order to achieve smaller and/or more granular compactions, the store files within a region are maintained separately for several row-key sub-ranges, or "stripes", of the region. The division is not visible to the higher levels of the system, so externally each region functions as before.
        This feature is fully compatible with default compactions - it can be enabled for existing tables, and the table will continue to operate normally if it's disabled later.

        When to use

        You might want to consider using this feature if you have:

        • large regions (in that case, you can get the positive effect of much smaller regions without additional memstore and region management overhead); or
        • non-uniform row keys, e.g. time dimension in a key (in that case, only the stripes receiving the new keys will keep compacting - old data will not compact as much, or at all).

        According to perf testing performed, in these case the read performance can improve somewhat, and the read and write performance variability due to compactions is greatly reduced. There's overall perf improvement on large, non-uniform row key regions (hash-prefixed timestamp key) over long term. All of these performance gains are best realized when table is already large. In future, the perf improvement might also extend to region splits.

        How to enable

        To use stripe compactions for a table or a column family, you should set its hbase.hstore.engine.class to org.apache.hadoop.hbase.regionserver.StripeStoreEngine. Due to the nature of compactions, you also need to set the blocking file count to a high number (100 is a good default, which is 10 times the normal default of 10). If changing the existing table, you should do it when it is disabled. Examples:

        alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}
        
        alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}
        
        create 'orders_table', 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}
        

        Then, you can configure the other options if needed (see below) and enable the table.
        To switch back to default compactions, set hbase.hstore.engine.class to nil to unset it; or set it explicitly to "org.apache.hadoop.hbase.regionserver.DefaultStoreEngine" (this also needs to be done on a disabled table).
        When you enable a large table after changing the store engine either way, a major compaction will likely be performed on most regions. This is not a problem with new tables.

        How to configure

        All of the settings described below are best set on table/cf level (with the table disabled first, for the settings to apply), similar to the above, e.g.

        alter 'orders_table', CONFIGURATION => {'key' => 'value', ..., 'key' => 'value'}}
        

        Region and stripe sizing

        Based on your region sizing, you might want to also change your stripe sizing. By default, your new regions will start with one stripe. When the stripe is too big (16 memstore flushes size), on next compaction it will be split into two stripes. Stripe splitting will continue in a similar manner as the region grows, until the region itself is big enough to split (region split will work the same as with default compactions).
        You can improve this pattern for your data. You should generally aim at stripe size of at least 1Gb, and about 8-12 stripes for uniform row keys - so, for example if your regions are 30 Gb, 12x2.5Gb stripes might be a good idea.
        The settings are as follows:

        Setting Notes
        hbase.store.stripe.
        initialStripeCount
        Initial stripe count to create. You can use it as follows:
        • for relatively uniform row keys, if you know the approximate target number of stripes from the above, you can avoid some splitting overhead by starting w/several stripes (2, 5, 10...). Note that if the early data is not representative of overall row key distribution, this will not be as efficient.
        • for existing tables with lots of data, you can use this to pre-split stripes.
        • for e.g. hash-prefixed sequential keys, with more than one hash prefix per region, you know that some pre-splitting makes sense.
        hbase.store.stripe.
        sizeToSplit
        Maximum stripe size before it's split. You can use this in conjunction with the next setting to control target stripe size (sizeToSplit = splitPartsCount * target stripe size), according to the above sizing considerations.
        hbase.store.stripe.
        splitPartCount
        The number of new stripes to create when splitting one. The default is 2, and is good for most cases. For non-uniform row keys, you might experiment with increasing the number somewhat (3-4), to isolate the arriving updates into narrower slice of the region with just one split instead of several.

        Memstore sizing

        By default, the flush creates several files from one memstore, according to existing stripe boundaries and row keys to flush. This approach minimizes write amplification, but can be undesirable if memstore is small and there are many stripes (the files will be too small).
        In such cases, you can set hbase.store.stripe.compaction.flushToL0 to true. This will cause flush to create a single file instead; when at least hbase.store.stripe.compaction.minFilesL0 such files (by default, 4) accumulate, they will be compacted into striped files.

        Normal compaction configuration

        All the settings that apply to normal compactions (file size limits, etc.) apply to stripe compactions. The exception are min and max number of files, which are set to higher values by default because the files in stripes are smaller. To control these for stripe compactions, use hbase.store.stripe.compaction.minFiles and .maxFiles.

        Show
        sershe Sergey Shelukhin added a comment - Here's a preview... wdyt? Introduction Stripe compactions is an experimental feature added in HBase 0.98 which aims to improve compactions for large regions or non-uniformly distributed row keys. In order to achieve smaller and/or more granular compactions, the store files within a region are maintained separately for several row-key sub-ranges, or "stripes", of the region. The division is not visible to the higher levels of the system, so externally each region functions as before. This feature is fully compatible with default compactions - it can be enabled for existing tables, and the table will continue to operate normally if it's disabled later. When to use You might want to consider using this feature if you have: large regions (in that case, you can get the positive effect of much smaller regions without additional memstore and region management overhead); or non-uniform row keys, e.g. time dimension in a key (in that case, only the stripes receiving the new keys will keep compacting - old data will not compact as much, or at all). According to perf testing performed, in these case the read performance can improve somewhat, and the read and write performance variability due to compactions is greatly reduced. There's overall perf improvement on large, non-uniform row key regions (hash-prefixed timestamp key) over long term. All of these performance gains are best realized when table is already large. In future, the perf improvement might also extend to region splits. How to enable To use stripe compactions for a table or a column family, you should set its hbase.hstore.engine.class to org.apache.hadoop.hbase.regionserver.StripeStoreEngine . Due to the nature of compactions, you also need to set the blocking file count to a high number (100 is a good default, which is 10 times the normal default of 10). If changing the existing table, you should do it when it is disabled. Examples: alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'} alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}} create 'orders_table', 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'} Then, you can configure the other options if needed (see below) and enable the table. To switch back to default compactions, set hbase.hstore.engine.class to nil to unset it; or set it explicitly to " org.apache.hadoop.hbase.regionserver.DefaultStoreEngine " (this also needs to be done on a disabled table). When you enable a large table after changing the store engine either way, a major compaction will likely be performed on most regions. This is not a problem with new tables. How to configure All of the settings described below are best set on table/cf level (with the table disabled first, for the settings to apply), similar to the above, e.g. alter 'orders_table', CONFIGURATION => {'key' => 'value', ..., 'key' => 'value'}} Region and stripe sizing Based on your region sizing, you might want to also change your stripe sizing. By default, your new regions will start with one stripe. When the stripe is too big (16 memstore flushes size), on next compaction it will be split into two stripes. Stripe splitting will continue in a similar manner as the region grows, until the region itself is big enough to split (region split will work the same as with default compactions). You can improve this pattern for your data. You should generally aim at stripe size of at least 1Gb, and about 8-12 stripes for uniform row keys - so, for example if your regions are 30 Gb, 12x2.5Gb stripes might be a good idea. The settings are as follows: Setting Notes hbase.store.stripe. initialStripeCount Initial stripe count to create. You can use it as follows: for relatively uniform row keys, if you know the approximate target number of stripes from the above, you can avoid some splitting overhead by starting w/several stripes (2, 5, 10...). Note that if the early data is not representative of overall row key distribution, this will not be as efficient. for existing tables with lots of data, you can use this to pre-split stripes. for e.g. hash-prefixed sequential keys, with more than one hash prefix per region, you know that some pre-splitting makes sense. hbase.store.stripe. sizeToSplit Maximum stripe size before it's split. You can use this in conjunction with the next setting to control target stripe size (sizeToSplit = splitPartsCount * target stripe size), according to the above sizing considerations. hbase.store.stripe. splitPartCount The number of new stripes to create when splitting one. The default is 2, and is good for most cases. For non-uniform row keys, you might experiment with increasing the number somewhat (3-4), to isolate the arriving updates into narrower slice of the region with just one split instead of several. Memstore sizing By default, the flush creates several files from one memstore, according to existing stripe boundaries and row keys to flush. This approach minimizes write amplification, but can be undesirable if memstore is small and there are many stripes (the files will be too small). In such cases, you can set hbase.store.stripe.compaction.flushToL0 to true. This will cause flush to create a single file instead; when at least hbase.store.stripe.compaction.minFilesL0 such files (by default, 4) accumulate, they will be compacted into striped files. Normal compaction configuration All the settings that apply to normal compactions (file size limits, etc.) apply to stripe compactions. The exception are min and max number of files, which are set to higher values by default because the files in stripes are smaller. To control these for stripe compactions, use hbase.store.stripe.compaction.minFiles and .maxFiles .

          People

          • Assignee:
            sershe Sergey Shelukhin
            Reporter:
            sershe Sergey Shelukhin
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development