Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-11316

Expand info about compactions beyond HBASE-11120

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: Compaction, documentation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Round 2 - expand info about the algorithms, talk about stripe compaction, and talk more about configuration. Hopefully find some rules of thumb.

      1. addendum.txt
        4 kB
        stack
      2. ch9_compactions.pdf
        134 kB
        Misty Stanley-Jones
      3. HBASE-11316.patch
        63 kB
        Misty Stanley-Jones
      4. HBASE-11316-1.patch
        84 kB
        Misty Stanley-Jones
      5. HBASE-11316-2.patch
        67 kB
        Misty Stanley-Jones
      6. HBASE-11316-3.patch
        79 kB
        Misty Stanley-Jones
      7. HBASE-11316-4.patch
        105 kB
        Misty Stanley-Jones
      8. HBASE-11316-5.patch
        106 kB
        Misty Stanley-Jones
      9. HBASE-11316-6.patch
        107 kB
        Misty Stanley-Jones
      10. HBASE-11316-6-rebased.patch
        103 kB
        Misty Stanley-Jones
      11. HBASE-11316-6-rebased-v2.patch
        103 kB
        Misty Stanley-Jones
      12. HBASE-11316-7.patch
        104 kB
        Misty Stanley-Jones
      13. HBASE-11316-8.patch
        104 kB
        Misty Stanley-Jones
      14. HBASE-11316-9.patch
        103 kB
        Misty Stanley-Jones

        Issue Links

          Activity

          Hide
          misty Misty Stanley-Jones added a comment -

          The following changes:

          • Created separate procedures for ExploringCompactionPolicy and RatioBasedCompactionPolicy
          • Showed how to enable RatioBasedCompactionPolicy if you still want it
          • Rewriting, reorg, and some pretty-printing in the StripePolicy section

          Question - should I remove the old Compaction section? Is there anything in it aside from the ASCII art that I don't have?

          Show
          misty Misty Stanley-Jones added a comment - The following changes: Created separate procedures for ExploringCompactionPolicy and RatioBasedCompactionPolicy Showed how to enable RatioBasedCompactionPolicy if you still want it Rewriting, reorg, and some pretty-printing in the StripePolicy section Question - should I remove the old Compaction section? Is there anything in it aside from the ASCII art that I don't have?
          Hide
          misty Misty Stanley-Jones added a comment -

          Expanded information about tuning the different compaction parameters based on feedback from Lars George (can't at-mention him).

          Show
          misty Misty Stanley-Jones added a comment - Expanded information about tuning the different compaction parameters based on feedback from Lars George (can't at-mention him).
          Hide
          misty Misty Stanley-Jones added a comment -

          File size shrunk because I rebased – sorry.

          Show
          misty Misty Stanley-Jones added a comment - File size shrunk because I rebased – sorry.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12650004/HBASE-11316-2.patch
          against trunk revision .
          ATTACHMENT ID: 12650004

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + which resides within a Store. The number of StoreFiles increases over time. <firstterm>Compaction</firstterm> is an operation which reduces the number of
          + <systemitem>StoreFile</systemitem>. Minor compactions do not drop (filter out) deletes or expired
          + versions, because of potential side effects. See <xref linkend="compaction.and.deletes" /> and <xref
          + is a minor or major compaction) is called the <firstterm>compaction policy</firstterm>.</para>
          + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>.
          + <entry><para>The maximum number of files which will be selected for a single minor
          + <entry><para>There are two different thread pools for compactions, one for large
          + major compactions. This approach alleviates some of the challenges of larger regions.</para>
          + <para>Stripe compaction is fully compatible with <xref linkend="compaction" /> and works in
          + conjunction with either the ExploringCompactionPolicy or RatioBasedCompactionPolicy. It can be enabled for

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:

          -1 core zombie tests. There are 1 zombie test(s):

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12650004/HBASE-11316-2.patch against trunk revision . ATTACHMENT ID: 12650004 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + which resides within a Store. The number of StoreFiles increases over time. <firstterm>Compaction</firstterm> is an operation which reduces the number of + <systemitem>StoreFile</systemitem>. Minor compactions do not drop (filter out) deletes or expired + versions, because of potential side effects. See <xref linkend="compaction.and.deletes" /> and <xref + is a minor or major compaction) is called the <firstterm>compaction policy</firstterm>.</para> + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>. + <entry><para>The maximum number of files which will be selected for a single minor + <entry><para>There are two different thread pools for compactions, one for large + major compactions. This approach alleviates some of the challenges of larger regions.</para> + <para>Stripe compaction is fully compatible with <xref linkend="compaction" /> and works in + conjunction with either the ExploringCompactionPolicy or RatioBasedCompactionPolicy. It can be enabled for +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9753//console This message is automatically generated.
          Hide
          stack stack added a comment -

          Ugh. This no longer applies. Please rebase Misty Stanley-Jones I took a read through. It is very good. Minor is that a minor compaction could end up with one file when done (you say it can't) and you never edit hbase-default.xml. You add your configs to hbase-site.xml.

          The writeups you have on each of the compaction configs, should these go into the actual hbase-default.xml too?

          Excellent Misty.

          Show
          stack stack added a comment - Ugh. This no longer applies. Please rebase Misty Stanley-Jones I took a read through. It is very good. Minor is that a minor compaction could end up with one file when done (you say it can't) and you never edit hbase-default.xml. You add your configs to hbase-site.xml. The writeups you have on each of the compaction configs, should these go into the actual hbase-default.xml too? Excellent Misty.
          Hide
          misty Misty Stanley-Jones added a comment -

          Rebased, fixed some conflicts. Fixed the nit. Did some small amount of rewording. Added missing settings to hbase-default.xml and did some reworking of existing descriptions in hbase-default.xml.

          Show
          misty Misty Stanley-Jones added a comment - Rebased, fixed some conflicts. Fixed the nit. Did some small amount of rewording. Added missing settings to hbase-default.xml and did some reworking of existing descriptions in hbase-default.xml.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12650499/HBASE-11316-3.patch
          against trunk revision .
          ATTACHMENT ID: 12650499

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + is a minor or major compaction) is called the <firstterm>compaction policy</firstterm>.</para>
          + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>.
          + major compactions. This approach alleviates some of the challenges of larger regions.</para>
          + <para>Stripe compaction is fully compatible with <xref linkend="compaction" /> and works in
          + conjunction with either the ExploringCompactionPolicy or RatioBasedCompactionPolicy. It can be enabled for
          + <para>You can enable stripe compaction for a table or a column family, by setting its <varname>hbase.hstore.engine.class</varname> to
          +<userinput>alter 'orders_table', CONFIGURATION =>

          {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}</userinput>
          +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput>
          +<userinput>create 'orders_table', 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}

          </userinput>
          + <varname>hbase.store.stripe.compaction.flushToL0</varname> to <literal>true</literal>. This will

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
          org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12650499/HBASE-11316-3.patch against trunk revision . ATTACHMENT ID: 12650499 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. -1 findbugs . The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + is a minor or major compaction) is called the <firstterm>compaction policy</firstterm>.</para> + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>. + major compactions. This approach alleviates some of the challenges of larger regions.</para> + <para>Stripe compaction is fully compatible with <xref linkend="compaction" /> and works in + conjunction with either the ExploringCompactionPolicy or RatioBasedCompactionPolicy. It can be enabled for + <para>You can enable stripe compaction for a table or a column family, by setting its <varname>hbase.hstore.engine.class</varname> to +<userinput>alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}</userinput> +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput> +<userinput>create 'orders_table', 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'} </userinput> + <varname>hbase.store.stripe.compaction.flushToL0</varname> to <literal>true</literal>. This will +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9776//console This message is automatically generated.
          Hide
          larsgeorge Lars George added a comment -

          Typo in "agiven" (missing space) and "milliseconds.." (two dots). I need to read the rest still, but that will take a little longer. Maybe this is larger enough to stick it on review board - or else it is getting messy feedback wise.

          Show
          larsgeorge Lars George added a comment - Typo in "agiven" (missing space) and "milliseconds.." (two dots). I need to read the rest still, but that will take a little longer. Maybe this is larger enough to stick it on review board - or else it is getting messy feedback wise.
          Hide
          misty Misty Stanley-Jones added a comment -

          I can't find either of your typos and am a bit worried you might be looking at an old output. Let me generate a PDF and attach. I don't know about Review Board – who should I ask?

          Show
          misty Misty Stanley-Jones added a comment - I can't find either of your typos and am a bit worried you might be looking at an old output. Let me generate a PDF and attach. I don't know about Review Board – who should I ask?
          Hide
          misty Misty Stanley-Jones added a comment -

          Printed a PDF showing the new content, for easier review.

          Show
          misty Misty Stanley-Jones added a comment - Printed a PDF showing the new content, for easier review.
          Hide
          misty Misty Stanley-Jones added a comment -

          BTW, noticed and fixed a problem (empty parentheses in the first line of the section) but won't regenerate the PDF for that.

          Show
          misty Misty Stanley-Jones added a comment - BTW, noticed and fixed a problem (empty parentheses in the first line of the section) but won't regenerate the PDF for that.
          Hide
          stack stack added a comment -

          I see 'agiven' in first description section in the hbase-default.xml. I started reviewing and it looks really good – thanks in particular for working on these hbase-default.xml descriptions – but I kept tripping over your use of HStore vs Store vs StoreFile vs HStoreFile. I started to edit and was adding in after StoreFile (HFile) in commas because that is how we usually refer to these files and Column Family for Store since they equate.... but I gave up thinking I should be punting this to you since you are probably trying to have one voice across the doc and one set of terminology you'd like to apply. How would you like to deal with HStoreFile vs StoreFile vs HFile? And Store vs HStore vs Column Family? Good on you Misty.

          Show
          stack stack added a comment - I see 'agiven' in first description section in the hbase-default.xml. I started reviewing and it looks really good – thanks in particular for working on these hbase-default.xml descriptions – but I kept tripping over your use of HStore vs Store vs StoreFile vs HStoreFile. I started to edit and was adding in after StoreFile (HFile) in commas because that is how we usually refer to these files and Column Family for Store since they equate.... but I gave up thinking I should be punting this to you since you are probably trying to have one voice across the doc and one set of terminology you'd like to apply. How would you like to deal with HStoreFile vs StoreFile vs HFile? And Store vs HStore vs Column Family? Good on you Misty.
          Hide
          misty Misty Stanley-Jones added a comment -

          Working on this today. Do you prefer HFile or StoreFile? I didn't actually realize they were the same thing when I was originally writing this.

          Show
          misty Misty Stanley-Jones added a comment - Working on this today. Do you prefer HFile or StoreFile? I didn't actually realize they were the same thing when I was originally writing this.
          Hide
          misty Misty Stanley-Jones added a comment -
          • Fixed typos I introduced into hbase-default.xml
          • Fixed some invalid XML that was introduced by other patches and came in via a rebase
          • Tried to unify terminology but I need clarification here. The old compaction docs used StoreFile, but stack's previous comment seemed to imply that he preferred HFile. Same thing with Store vs ColumnFamily. It's weird to say that a ColumnFamily has HFiles. Isn't it more that the HFiles represent the ColumnFamily's data? Anyway I played around with this and tried to be consistent in using HFile instead of StoreFile, Column Family instead of Store. This is easy to revert if I need to – please give me your opinions.
          Show
          misty Misty Stanley-Jones added a comment - Fixed typos I introduced into hbase-default.xml Fixed some invalid XML that was introduced by other patches and came in via a rebase Tried to unify terminology but I need clarification here. The old compaction docs used StoreFile, but stack 's previous comment seemed to imply that he preferred HFile. Same thing with Store vs ColumnFamily. It's weird to say that a ColumnFamily has HFiles. Isn't it more that the HFiles represent the ColumnFamily's data? Anyway I played around with this and tried to be consistent in using HFile instead of StoreFile, Column Family instead of Store. This is easy to revert if I need to – please give me your opinions.
          Hide
          misty Misty Stanley-Jones added a comment -

          Got clarification from JD that it is most appropriate to talk about StoreFiles and Stores, not HFiles and ColumnFamilies. I'll update that patch to reflect that. Sorry about the churn.

          Show
          misty Misty Stanley-Jones added a comment - Got clarification from JD that it is most appropriate to talk about StoreFiles and Stores, not HFiles and ColumnFamilies. I'll update that patch to reflect that. Sorry about the churn.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12653289/HBASE-11316-4.patch
          against trunk revision .
          ATTACHMENT ID: 12653289

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + <description> The time for which a region will block updates after reaching the StoreFile (HFile)
          + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description>
          + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>.
          + compaction. This may be because not all HFiles in the Column Family are available to
          + less than the sum of the sizes of the other HFiles in the set.</para>
          + HFile. Its function is to remove deletes and expired versions, and reset locality
          + HFile to be compacted, a major compaction is run, even if it would otherwise have
          + compaction. Its effect is to limit compaction of large HFiles. The value of
          + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store
          +<userinput>alter 'orders_table', CONFIGURATION =>

          {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}

          </userinput>

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
          org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12653289/HBASE-11316-4.patch against trunk revision . ATTACHMENT ID: 12653289 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. -1 findbugs . The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + <description> The time for which a region will block updates after reaching the StoreFile (HFile) + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description> + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>. + compaction. This may be because not all HFiles in the Column Family are available to + less than the sum of the sizes of the other HFiles in the set.</para> + HFile. Its function is to remove deletes and expired versions, and reset locality + HFile to be compacted, a major compaction is run, even if it would otherwise have + compaction. Its effect is to limit compaction of large HFiles. The value of + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store +<userinput>alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'} </userinput> +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9908//console This message is automatically generated.
          Hide
          misty Misty Stanley-Jones added a comment -

          Ready for review (again).

          Show
          misty Misty Stanley-Jones added a comment - Ready for review (again).
          Hide
          misty Misty Stanley-Jones added a comment -

          PDF attached for easier review – did not include the generated hbase-default.xml contents.

          Show
          misty Misty Stanley-Jones added a comment - PDF attached for easier review – did not include the generated hbase-default.xml contents.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12653316/HBASE-11316-5.patch
          against trunk revision .
          ATTACHMENT ID: 12653316

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description>
          + drop (filter out) deletes or expired versions, because of potential side effects. See <xref
          + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>.
          + will compact larger StoreFiles. However, during reads, HBase will need to seek
          + through fewer StpreFo;es to accomplish the read. Consider this approach if you
          + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store
          +<userinput>alter 'orders_table', CONFIGURATION =>

          {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}</userinput>
          +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput>
          +<userinput>create 'orders_table', 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}

          </userinput>

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
          org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12653316/HBASE-11316-5.patch against trunk revision . ATTACHMENT ID: 12653316 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. -1 findbugs . The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description> + drop (filter out) deletes or expired versions, because of potential side effects. See <xref + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>. + will compact larger StoreFiles. However, during reads, HBase will need to seek + through fewer StpreFo;es to accomplish the read. Consider this approach if you + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store +<userinput>alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}</userinput> +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput> +<userinput>create 'orders_table', 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'} </userinput> +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9912//console This message is automatically generated.
          Hide
          stack stack added a comment -

          Anyway I played around with this and tried to be consistent in using HFile instead of StoreFile, Column Family instead of Store.

          This is the way to go IMO.

          Looking at PDF, you have this in spite of the above: "it flushes its contents to a StoreFile. The number of StoreFiles in a Store increases over time. " What you thinking given the above?

          Show
          stack stack added a comment - Anyway I played around with this and tried to be consistent in using HFile instead of StoreFile, Column Family instead of Store. This is the way to go IMO. Looking at PDF, you have this in spite of the above: "it flushes its contents to a StoreFile. The number of StoreFiles in a Store increases over time. " What you thinking given the above?
          Hide
          misty Misty Stanley-Jones added a comment -

          So, Jean-Daniel Cryans, Esteban Gutierrez, and I talked about this yesterday and he seemed to have the opinion that we should be using StoreFile and Store, since we could support other types of StoreFiles besides HFile later on. Based on that, I changed it all back. Now I will hold off on further changes until I get more consistent advice about how I Should be referring to these things.

          Show
          misty Misty Stanley-Jones added a comment - So, Jean-Daniel Cryans , Esteban Gutierrez , and I talked about this yesterday and he seemed to have the opinion that we should be using StoreFile and Store, since we could support other types of StoreFiles besides HFile later on. Based on that, I changed it all back. Now I will hold off on further changes until I get more consistent advice about how I Should be referring to these things.
          Hide
          stack stack added a comment -

          Folks seem to update hfile rather than put in place a new version (it is on version 3 at the moment). hfile seems to be known outside hbase (we bulk import hfiles, not storefiles).

          hfile is used more often in our doc currently:

          kalashnikov:docbkx stack$ grep hfile *|wc
          97 980 11524
          kalashnikov:docbkx stack$ grep -i hfile * |wc
          252 4413 35218
          kalashnikov:docbkx stack$ grep -i storefile * |wc
          188 2386 22833

          If you just use storefile, folks will wonder where hfile fits.

          You are the boss. Pick one. Just explain how the two relate on first use. Good on you misty. Ditto with column family/Store.

          Show
          stack stack added a comment - Folks seem to update hfile rather than put in place a new version (it is on version 3 at the moment). hfile seems to be known outside hbase (we bulk import hfiles, not storefiles). hfile is used more often in our doc currently: kalashnikov:docbkx stack$ grep hfile *|wc 97 980 11524 kalashnikov:docbkx stack$ grep -i hfile * |wc 252 4413 35218 kalashnikov:docbkx stack$ grep -i storefile * |wc 188 2386 22833 If you just use storefile, folks will wonder where hfile fits. You are the boss. Pick one. Just explain how the two relate on first use. Good on you misty. Ditto with column family/Store.
          Hide
          misty Misty Stanley-Jones added a comment -

          OK, then how do they relate? Especially Store / Column Family. It seems really strange to me to talk about an HFile being "in" a Column Family. Can you help me out?

          Show
          misty Misty Stanley-Jones added a comment - OK, then how do they relate? Especially Store / Column Family. It seems really strange to me to talk about an HFile being "in" a Column Family. Can you help me out?
          Hide
          stack stack added a comment -

          In the code, StoreFile is a facade over our file implementation HFile. Store is how we implement the modeling concept column family. We should have named Stores 'ColumnFamily' rather than Store.

          Show
          stack stack added a comment - In the code, StoreFile is a facade over our file implementation HFile. Store is how we implement the modeling concept column family. We should have named Stores 'ColumnFamily' rather than Store.
          Hide
          jdcryans Jean-Daniel Cryans added a comment -

          IMO, unless we specifically refer to something related to the HFile implementation, we should always use StoreFile. That's also how we present in the web UI when list the number of Stores and StoreFiles in a region.

          Show
          jdcryans Jean-Daniel Cryans added a comment - IMO, unless we specifically refer to something related to the HFile implementation, we should always use StoreFile. That's also how we present in the web UI when list the number of Stores and StoreFiles in a region.
          Hide
          stack stack added a comment -

          Misty you have enough to go on? Just ask if not.

          Show
          stack stack added a comment - Misty you have enough to go on? Just ask if not.
          Hide
          misty Misty Stanley-Jones added a comment -

          Well, it seems like you and JD are in disagreement on this so I am unsure how to proceed.

          Show
          misty Misty Stanley-Jones added a comment - Well, it seems like you and JD are in disagreement on this so I am unsure how to proceed.
          Hide
          misty Misty Stanley-Jones added a comment -

          Added a little explanation of StoreFile / HFile and Store/ColumnFamily at the beginning of the section on compaction!

          Show
          misty Misty Stanley-Jones added a comment - Added a little explanation of StoreFile / HFile and Store/ColumnFamily at the beginning of the section on compaction!
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12653533/HBASE-11316-6.patch
          against trunk revision .
          ATTACHMENT ID: 12653533

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description>
          + <listitem><para>A <firstterm>StoreFile</firstterm> is a facade of HFile. In terms of compaction, use of
          + xlink:href="https://issues.apache.org/jira/browse/HBASE-11316">HBASE-11316</link>.</para>
          + drop (filter out) deletes or expired versions, because of potential side effects. See <xref
          + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>.
          + will compact larger StoreFiles. However, during reads, HBase will need to seek
          + through fewer StpreFo;es to accomplish the read. Consider this approach if you
          + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store
          +<userinput>alter 'orders_table', CONFIGURATION =>

          {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}

          </userinput>
          +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput>

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
          org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12653533/HBASE-11316-6.patch against trunk revision . ATTACHMENT ID: 12653533 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. -1 findbugs . The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description> + <listitem><para>A <firstterm>StoreFile</firstterm> is a facade of HFile. In terms of compaction, use of + xlink:href="https://issues.apache.org/jira/browse/HBASE-11316"> HBASE-11316 </link>.</para> + drop (filter out) deletes or expired versions, because of potential side effects. See <xref + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>. + will compact larger StoreFiles. However, during reads, HBase will need to seek + through fewer StpreFo;es to accomplish the read. Consider this approach if you + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store +<userinput>alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'} </userinput> +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput> +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9936//console This message is automatically generated.
          Hide
          stack stack added a comment -

          Well, it seems like you and JD are in disagreement on this so I am unsure how to proceed.

          Sorry for confusion caused. Go storefile and store. JD and Esteban have a point especially given that is how we talk up files in UI (metrics also talk storefile). Suggest talking up how relates to hfile and column family since these are notions that are all over the hbase shop too. Thanks Misty.

          Show
          stack stack added a comment - Well, it seems like you and JD are in disagreement on this so I am unsure how to proceed. Sorry for confusion caused. Go storefile and store. JD and Esteban have a point especially given that is how we talk up files in UI (metrics also talk storefile). Suggest talking up how relates to hfile and column family since these are notions that are all over the hbase shop too. Thanks Misty.
          Hide
          misty Misty Stanley-Jones added a comment -

          Had to rebase.

          Show
          misty Misty Stanley-Jones added a comment - Had to rebase.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12654913/HBASE-11316-6-rebased.patch
          against trunk revision .
          ATTACHMENT ID: 12654913

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10008//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12654913/HBASE-11316-6-rebased.patch against trunk revision . ATTACHMENT ID: 12654913 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10008//console This message is automatically generated.
          Hide
          misty Misty Stanley-Jones added a comment -

          Rebased again, no content changed, just keeping it fresh.

          Show
          misty Misty Stanley-Jones added a comment - Rebased again, no content changed, just keeping it fresh.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12656404/HBASE-11316-6-rebased-v2.patch
          against trunk revision .
          ATTACHMENT ID: 12656404

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description>
          + <listitem><para>A <firstterm>StoreFile</firstterm> is a facade of HFile. In terms of compaction, use of
          + xlink:href="https://issues.apache.org/jira/browse/HBASE-11316">HBASE-11316</link>.</para>
          + drop (filter out) deletes or expired versions, because of potential side effects. See <xref
          + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>.
          + will compact larger StoreFiles. However, during reads, HBase will need to seek
          + through fewer StpreFo;es to accomplish the read. Consider this approach if you
          + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store
          +<userinput>alter 'orders_table', CONFIGURATION =>

          {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}

          </userinput>
          +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput>

          -1 site. The patch appears to cause mvn site goal to fail.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection
          org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12656404/HBASE-11316-6-rebased-v2.patch against trunk revision . ATTACHMENT ID: 12656404 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description> + <listitem><para>A <firstterm>StoreFile</firstterm> is a facade of HFile. In terms of compaction, use of + xlink:href="https://issues.apache.org/jira/browse/HBASE-11316"> HBASE-11316 </link>.</para> + drop (filter out) deletes or expired versions, because of potential side effects. See <xref + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>. + will compact larger StoreFiles. However, during reads, HBase will need to seek + through fewer StpreFo;es to accomplish the read. Consider this approach if you + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store +<userinput>alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'} </userinput> +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput> -1 site . The patch appears to cause mvn site goal to fail. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10113//console This message is automatically generated.
          Hide
          misty Misty Stanley-Jones added a comment -

          The last patch broke everything, don't use it. Stand by for a fix.

          Show
          misty Misty Stanley-Jones added a comment - The last patch broke everything, don't use it. Stand by for a fix.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12656417/HBASE-11316-7.patch
          against trunk revision .
          ATTACHMENT ID: 12656417

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description>
          + <listitem><para>A <firstterm>StoreFile</firstterm> is a facade of HFile. In terms of compaction, use of
          + xlink:href="https://issues.apache.org/jira/browse/HBASE-11316">HBASE-11316</link>.</para>
          + drop (filter out) deletes or expired versions, because of potential side effects. See <xref
          + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>.
          + will compact larger StoreFiles. However, during reads, HBase will need to seek
          + through fewer StpreFo;es to accomplish the read. Consider this approach if you
          + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store
          +<userinput>alter 'orders_table', CONFIGURATION =>

          {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}

          </userinput>
          +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput>

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
          org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12656417/HBASE-11316-7.patch against trunk revision . ATTACHMENT ID: 12656417 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description> + <listitem><para>A <firstterm>StoreFile</firstterm> is a facade of HFile. In terms of compaction, use of + xlink:href="https://issues.apache.org/jira/browse/HBASE-11316"> HBASE-11316 </link>.</para> + drop (filter out) deletes or expired versions, because of potential side effects. See <xref + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>. + will compact larger StoreFiles. However, during reads, HBase will need to seek + through fewer StpreFo;es to accomplish the read. Consider this approach if you + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store +<userinput>alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'} </userinput> +<userinput>alter 'orders_table', {NAME => 'blobs_cf', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}}</userinput> +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10115//console This message is automatically generated.
          Hide
          misty Misty Stanley-Jones added a comment -

          Had to rebase, this builds cleanly.

          Show
          misty Misty Stanley-Jones added a comment - Had to rebase, this builds cleanly.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12658042/HBASE-11316-8.patch
          against trunk revision .
          ATTACHMENT ID: 12658042

          +1 @author. The patch does not contain any @author tags.

          +0 tests included. The patch appears to be a documentation patch that doesn't require tests.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 lineLengths. The patch introduces the following lines longer than 100:
          + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description>
          + HBase since version 1.0 and ask that a column family have both its meta and DATA blocks hosted onheap in the L1 tier by
          + <listitem><para>A <firstterm>StoreFile</firstterm> is a facade of HFile. In terms of compaction, use of
          + xlink:href="https://issues.apache.org/jira/browse/HBASE-11316">HBASE-11316</link>.</para>
          + drop (filter out) deletes or expired versions, because of potential side effects. See <xref
          + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>.
          + will compact larger StoreFiles. However, during reads, HBase will need to seek
          + through fewer StpreFo;es to accomplish the read. Consider this approach if you
          + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store
          +<userinput>alter 'orders_table', CONFIGURATION =>

          {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'}

          </userinput>

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
          org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12658042/HBASE-11316-8.patch against trunk revision . ATTACHMENT ID: 12658042 +1 @author . The patch does not contain any @author tags. +0 tests included . The patch appears to be a documentation patch that doesn't require tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 lineLengths . The patch introduces the following lines longer than 100: + can try raising this value. Default: the value of LONG.MAX_VALUE, expressed in bytes.</description> + HBase since version 1.0 and ask that a column family have both its meta and DATA blocks hosted onheap in the L1 tier by + <listitem><para>A <firstterm>StoreFile</firstterm> is a facade of HFile. In terms of compaction, use of + xlink:href="https://issues.apache.org/jira/browse/HBASE-11316"> HBASE-11316 </link>.</para> + drop (filter out) deletes or expired versions, because of potential side effects. See <xref + <filename>hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/ExploringCompactionPolicy.java</filename>. + will compact larger StoreFiles. However, during reads, HBase will need to seek + through fewer StpreFo;es to accomplish the read. Consider this approach if you + xlink:href="http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/Store.html#836">Store +<userinput>alter 'orders_table', CONFIGURATION => {'hbase.hstore.engine.class' => 'org.apache.hadoop.hbase.regionserver.StripeStoreEngine', 'hbase.hstore.blockingStoreFiles' => '100'} </userinput> +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10197//console This message is automatically generated.
          Hide
          jdcryans Jean-Daniel Cryans added a comment -

          Comments:

          -    <description>Name of the failsafe snapshot taken by the restore operation.
          +    <description>Name of the failsafe snapshot taken by the restore opecompn.
          

          Typo slipped in there.

          +              <title>Being Stuck</title>
          

          Looking at the output, this line (and the other titles at that level) is comically small. There's probably we can do short of changing the style sheet.

          And I'm good with the rest.

          Show
          jdcryans Jean-Daniel Cryans added a comment - Comments: - <description>Name of the failsafe snapshot taken by the restore operation. + <description>Name of the failsafe snapshot taken by the restore opecompn. Typo slipped in there. + <title>Being Stuck</title> Looking at the output, this line (and the other titles at that level) is comically small. There's probably we can do short of changing the style sheet. And I'm good with the rest.
          Hide
          misty Misty Stanley-Jones added a comment -

          Fixed the typo. The size of the section titles is due to the level of nesting, which is just too deep but out of scope for this JIRA.

          Show
          misty Misty Stanley-Jones added a comment - Fixed the typo. The size of the section titles is due to the level of nesting, which is just too deep but out of scope for this JIRA.
          Hide
          jdcryans Jean-Daniel Cryans added a comment -

          Pushed to master, thanks for all the hard work documenting our most complicated stuff Misty.

          Show
          jdcryans Jean-Daniel Cryans added a comment - Pushed to master, thanks for all the hard work documenting our most complicated stuff Misty.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK #5349 (See https://builds.apache.org/job/HBase-TRUNK/5349/)
          HBASE-11316 Expand info about compactions beyond HBASE-11120 (Misty Stanley-Jones) (jdcryans: rev afae1e2583b8b32d828efbadce363247c784a93e)

          • src/main/docbkx/book.xml
          • hbase-common/src/main/resources/hbase-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5349 (See https://builds.apache.org/job/HBase-TRUNK/5349/ ) HBASE-11316 Expand info about compactions beyond HBASE-11120 (Misty Stanley-Jones) (jdcryans: rev afae1e2583b8b32d828efbadce363247c784a93e) src/main/docbkx/book.xml hbase-common/src/main/resources/hbase-default.xml
          Hide
          stack stack added a comment -

          Interesting. The test failures are legit:

          org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy
          org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection

          The hbase-default additions seem to break above tests. Let me isolate which changes are responsible for the break.

          Show
          stack stack added a comment - Interesting. The test failures are legit: org.apache.hadoop.hbase.regionserver.compactions.TestStripeCompactionPolicy org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection The hbase-default additions seem to break above tests. Let me isolate which changes are responsible for the break.
          Hide
          stack stack added a comment -

          Addendum. The two failing tests don't like the setting of explicit new default in hbase-default.xml of "hbase.hstore.compaction.min.size" so just unset this value so these tests pass again.

          Show
          stack stack added a comment - Addendum. The two failing tests don't like the setting of explicit new default in hbase-default.xml of "hbase.hstore.compaction.min.size" so just unset this value so these tests pass again.
          Hide
          jdcryans Jean-Daniel Cryans added a comment -

          +1, thanks for looking at this Stack and sorry I overlooked those failing tests.

          Show
          jdcryans Jean-Daniel Cryans added a comment - +1, thanks for looking at this Stack and sorry I overlooked those failing tests.
          Hide
          stack stack added a comment -

          Pushed addendum. Jean-Daniel Cryans now I only owe you $2.1455M dollars.

          Show
          stack stack added a comment - Pushed addendum. Jean-Daniel Cryans now I only owe you $2.1455M dollars.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK #5353 (See https://builds.apache.org/job/HBase-TRUNK/5353/)
          HBASE-11316 Expand info about compactions beyond HBASE-11120 ADDENDUM TO FIX BROKE TESTS (stack: rev 85853f69c88c08db61af7fb6ef3b5432f926e531)

          • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultCompactSelection.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5353 (See https://builds.apache.org/job/HBase-TRUNK/5353/ ) HBASE-11316 Expand info about compactions beyond HBASE-11120 ADDENDUM TO FIX BROKE TESTS (stack: rev 85853f69c88c08db61af7fb6ef3b5432f926e531) hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestDefaultCompactSelection.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/compactions/TestStripeCompactionPolicy.java

            People

            • Assignee:
              misty Misty Stanley-Jones
              Reporter:
              misty Misty Stanley-Jones
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development