Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.4, 0.95.0
    • Component/s: mapreduce
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Add Bloom Filter support for bulk imports. Lacking a bloom filter, even on a single imported file, can cause perf degradation. Since we now set our compression type based on the HBase CF configuration, it would be good to follow this path for the bloom filter addition.

      1. HBASE-3776.patch
        8 kB
        Anoop Sam John
      2. HBASE-3776_v2.patch
        8 kB
        Anoop Sam John
      3. 3776-trunk.addendum
        1.0 kB
        Ted Yu
      4. 3776-0.94.txt
        8 kB
        Lars Hofhansl

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          223d 4h 1m 1 Jerry Chen 22/Nov/11 05:04
          In Progress In Progress Patch Available Patch Available
          389d 2h 28m 1 Anoop Sam John 15/Dec/12 07:32
          Open Open Patch Available Patch Available
          19s 1 Anoop Sam John 15/Dec/12 13:13
          Patch Available Patch Available Open Open
          12d 22h 21m 2 Lars Hofhansl 28/Dec/12 05:53
          Open Open Resolved Resolved
          38m 25s 1 Lars Hofhansl 28/Dec/12 06:32
          Resolved Resolved Closed Closed
          15d 23h 56m 1 Lars Hofhansl 13/Jan/13 06:28
          Lars Hofhansl made changes -
          Fix Version/s 0.94.4 [ 12323367 ]
          Fix Version/s 0.94.0 [ 12316419 ]
          Lars Hofhansl made changes -
          Fix Version/s 0.94.0 [ 12316419 ]
          stack made changes -
          Fix Version/s 0.95.0 [ 12324094 ]
          Fix Version/s 0.96.0 [ 12320040 ]
          Fix Version/s 0.94.4 [ 12323367 ]
          Lars Hofhansl made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security-on-Hadoop-23 #10 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/10/)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #10 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/10/ ) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security #90 (See https://builds.apache.org/job/HBase-0.94-security/90/)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security #90 (See https://builds.apache.org/job/HBase-0.94-security/90/ ) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Ted Yu added a comment -

          The above comment was inaccurate: javac warning issue was due mvn command not found on hadoop9.

          Show
          Ted Yu added a comment - The above comment was inaccurate: javac warning issue was due mvn command not found on hadoop9.
          Hide
          Ted Yu added a comment -

          looks like this Jira is related to the javac warning issue in recent PreCommit builds.

          Show
          Ted Yu added a comment - looks like this Jira is related to the javac warning issue in recent PreCommit builds.
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #316 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/316/)
          HBASE-3776 Addendum fixes reference to BloomType (Revision 1426409)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426406)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java

          tedyu :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #316 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/316/ ) HBASE-3776 Addendum fixes reference to BloomType (Revision 1426409) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426406) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #673 (See https://builds.apache.org/job/HBase-0.94/673/)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415)

          Result = FAILURE
          larsh :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #673 (See https://builds.apache.org/job/HBase-0.94/673/ ) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426415) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #3665 (See https://builds.apache.org/job/HBase-TRUNK/3665/)
          HBASE-3776 Addendum fixes reference to BloomType (Revision 1426409)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #3665 (See https://builds.apache.org/job/HBase-TRUNK/3665/ ) HBASE-3776 Addendum fixes reference to BloomType (Revision 1426409) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Lars Hofhansl made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Lars Hofhansl added a comment -

          Committed to 0.94 as well. Thanks for the patch Anoop.

          Show
          Lars Hofhansl added a comment - Committed to 0.94 as well. Thanks for the patch Anoop.
          Hide
          Lars Hofhansl added a comment -

          I ran TestHFileOutputFormat in 0.94.
          Patch's almost identical to trunks, committing soon, unless anybody objects.

          Show
          Lars Hofhansl added a comment - I ran TestHFileOutputFormat in 0.94. Patch's almost identical to trunks, committing soon, unless anybody objects.
          Lars Hofhansl made changes -
          Fix Version/s 0.94.4 [ 12323367 ]
          Hide
          Anoop Sam John added a comment -

          Thanks for that addendum Ted.
          Lars thanks for the port. +1

          Show
          Anoop Sam John added a comment - Thanks for that addendum Ted. Lars thanks for the port. +1
          Hide
          Lars Hofhansl added a comment -

          Opps @Anop. Just saw you latest comment now. Anyway, please review the 0.94 patch I attached.

          Show
          Lars Hofhansl added a comment - Opps @Anop. Just saw you latest comment now. Anyway, please review the 0.94 patch I attached.
          Lars Hofhansl made changes -
          Attachment 3776-0.94.txt [ 12562560 ]
          Hide
          Lars Hofhansl added a comment -

          Here's a 0.94 patch.

          Show
          Lars Hofhansl added a comment - Here's a 0.94 patch.
          Lars Hofhansl made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Ted Yu made changes -
          Attachment 3776-trunk.addendum [ 12562558 ]
          Hide
          Ted Yu added a comment -

          Addendum for trunk that fixes reference to BloomType

          Show
          Ted Yu added a comment - Addendum for trunk that fixes reference to BloomType
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #3664 (See https://builds.apache.org/job/HBase-TRUNK/3664/)
          HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426406)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #3664 (See https://builds.apache.org/job/HBase-TRUNK/3664/ ) HBASE-3776 Add Bloom Filter Support to HFileOutputFormat (Anoop Sam John) (Revision 1426406) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java
          Hide
          Ted Yu added a comment -

          Integrated to trunk.

          Thanks for the patch, Anoop.

          Thanks for the review, Lars.

          Show
          Ted Yu added a comment - Integrated to trunk. Thanks for the patch, Anoop. Thanks for the review, Lars.
          Hide
          Anoop Sam John added a comment -

          Lars
          I can give a patch for 94 version also. Will upload tonight..

          Show
          Anoop Sam John added a comment - Lars I can give a patch for 94 version also. Will upload tonight..
          Hide
          Lars Hofhansl added a comment -

          Patch looks good to me +1
          I think we should have this in 0.94 as well.

          Show
          Lars Hofhansl added a comment - Patch looks good to me +1 I think we should have this in 0.94 as well.
          Hide
          Ted Yu added a comment -

          I already gave my +1.
          One more +1 is needed.

          Show
          Ted Yu added a comment - I already gave my +1. One more +1 is needed.
          Hide
          Anoop Sam John added a comment -

          @Ted, @stack - Is it good to go?

          Show
          Anoop Sam John added a comment - @Ted, @stack - Is it good to go?
          Ted Yu made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Hide
          Ted Yu added a comment -

          Test failure in TestMultiParallel is not related to the patch.

          +1 on v2.

          Show
          Ted Yu added a comment - Test failure in TestMultiParallel is not related to the patch. +1 on v2.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12561114/HBASE-3776_v2.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any additional warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestMultiParallel

          -1 core zombie tests. There are zombie tests. See build logs for details.

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561114/HBASE-3776_v2.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any additional warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestMultiParallel -1 core zombie tests . There are zombie tests. See build logs for details. Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3562//console This message is automatically generated.
          Anoop Sam John made changes -
          Component/s mapreduce [ 12313047 ]
          Anoop Sam John made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Anoop Sam John made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Anoop Sam John made changes -
          Attachment HBASE-3776_v2.patch [ 12561114 ]
          Hide
          Anoop Sam John added a comment -

          Addressing Ted's comments

          Show
          Anoop Sam John added a comment - Addressing Ted's comments
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12561005/HBASE-3776.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any additional warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestMultiParallel

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12561005/HBASE-3776.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any additional warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestMultiParallel Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3560//console This message is automatically generated.
          Hide
          Anoop Sam John added a comment -

          Submit patch. Let QA run the tests

          Show
          Anoop Sam John added a comment - Submit patch. Let QA run the tests
          Anoop Sam John made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Hide
          Anoop Sam John added a comment -

          I will give updated patch today addressing Ted's comments

          Show
          Anoop Sam John added a comment - I will give updated patch today addressing Ted's comments
          Hide
          Ted Yu added a comment -

          Patch looks good.
          TestHFileOutputFormat passes.
          Some minor comments below.

          +  private static Map<byte[], String> createFamilyConfValueMap(Configuration conf, String confName) {
          

          Please add javadoc for the above method.

          +    StringBuilder bloomTypeConfigValue = new StringBuilder();
          +    HTableDescriptor tableDescriptor = table.getTableDescriptor();
          +    if (tableDescriptor == null) {
          +      // could happen with mock table instance
          +      return;
          

          Please move StringBuilder ctor to after the above if block.

          Show
          Ted Yu added a comment - Patch looks good. TestHFileOutputFormat passes. Some minor comments below. + private static Map< byte [], String > createFamilyConfValueMap(Configuration conf, String confName) { Please add javadoc for the above method. + StringBuilder bloomTypeConfigValue = new StringBuilder(); + HTableDescriptor tableDescriptor = table.getTableDescriptor(); + if (tableDescriptor == null ) { + // could happen with mock table instance + return ; Please move StringBuilder ctor to after the above if block.
          Jerry Chen made changes -
          Assignee Jerry Chen [ gqchen ] Anoop Sam John [ anoopsamjohn ]
          Hide
          Anoop Sam John added a comment -

          HFileOutputFormat creates StoreFile writer now so that the handling of the blooms will happen within StoreFile

          Show
          Anoop Sam John added a comment - HFileOutputFormat creates StoreFile writer now so that the handling of the blooms will happen within StoreFile
          Anoop Sam John made changes -
          Attachment HBASE-3776.patch [ 12561005 ]
          Hide
          Anoop Sam John added a comment -

          Started working with this. Will give patch next week after test.

          Show
          Anoop Sam John added a comment - Started working with this. Will give patch next week after test.
          Hide
          Jerry Chen added a comment -

          Please. Sorry I wasn't able to get to this.

          Sent from my iPhone

          Show
          Jerry Chen added a comment - Please. Sorry I wasn't able to get to this. Sent from my iPhone
          Hide
          Anoop Sam John added a comment -

          @Jerry if you are not working on this , I can take a look at this.

          Show
          Anoop Sam John added a comment - @Jerry if you are not working on this , I can take a look at this.
          Hide
          Anoop Sam John added a comment -

          Here the issue was handling of the blooms was happening at the StoreFile level not at the HFile level. The HFileOutputFormat deals with HFiles directly and creates the HFile writers. Can HFileOutputFormat be a StoreFileOutputFormat?
          Just we need to pass the bloom type and the rest will be handled.

          Show
          Anoop Sam John added a comment - Here the issue was handling of the blooms was happening at the StoreFile level not at the HFile level. The HFileOutputFormat deals with HFiles directly and creates the HFile writers. Can HFileOutputFormat be a StoreFileOutputFormat? Just we need to pass the bloom type and the rest will be handled.
          Hide
          Chip Salzenberg added a comment -

          Hello? Anyone? Bueller?

          Show
          Chip Salzenberg added a comment - Hello? Anyone? Bueller?
          Hide
          Anoop Sam John added a comment -

          @Jerry Any work happening regarding this?

          Show
          Anoop Sam John added a comment - @Jerry Any work happening regarding this?
          Lars Hofhansl made changes -
          Priority Blocker [ 1 ] Critical [ 2 ]
          Lars Hofhansl made changes -
          Fix Version/s 0.96.0 [ 12320040 ]
          Fix Version/s 0.94.0 [ 12316419 ]
          Hide
          Lars Hofhansl added a comment -

          I don't buy that this is a 0.94 blocker. Pull back if you disagree.

          Show
          Lars Hofhansl added a comment - I don't buy that this is a 0.94 blocker. Pull back if you disagree.
          Hide
          Lars Hofhansl added a comment -

          Please comment today on why this is a blocker for 0.94. Otherwise I'll move this out of 0.94.

          Show
          Lars Hofhansl added a comment - Please comment today on why this is a blocker for 0.94. Otherwise I'll move this out of 0.94.
          Hide
          Lars Hofhansl added a comment -

          Why is this a 0.94 blocker? We lives with this in 0.92.

          Show
          Lars Hofhansl added a comment - Why is this a 0.94 blocker? We lives with this in 0.92.
          stack made changes -
          Fix Version/s 0.94.0 [ 12316419 ]
          Fix Version/s 0.96.0 [ 12320040 ]
          Hide
          stack added a comment -

          Misread. N said 0.94 blocker.

          Show
          stack added a comment - Misread. N said 0.94 blocker.
          stack made changes -
          Fix Version/s 0.96.0 [ 12320040 ]
          Priority Minor [ 4 ] Blocker [ 1 ]
          Hide
          stack added a comment -

          Making blocker on 0.96 at Nicolas's suggestion

          Show
          stack added a comment - Making blocker on 0.96 at Nicolas's suggestion
          Nicolas Spiegelberg made changes -
          Parent HBASE-4907 [ 12533357 ]
          Issue Type Improvement [ 4 ] Sub-task [ 7 ]
          Jerry Chen made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Jerry Chen made changes -
          Assignee Mikhail Bautin [ mikhail ] Jerry Chen [ gqchen ]
          Hide
          stack added a comment -

          Moving out of 0.92. Move it back in if you think differently.

          Show
          stack added a comment - Moving out of 0.92. Move it back in if you think differently.
          stack made changes -
          Fix Version/s 0.92.0 [ 12314223 ]
          Hide
          stack added a comment -

          Moving out of 0.92. Move it back in if you think differently.

          Show
          stack added a comment - Moving out of 0.92. Move it back in if you think differently.
          Hide
          Nicolas Spiegelberg added a comment -

          HBASE-3763 required to easily support HFOF, since we will no longer need to know exact max key count before creating an HFile.

          Show
          Nicolas Spiegelberg added a comment - HBASE-3763 required to easily support HFOF, since we will no longer need to know exact max key count before creating an HFile.
          Nicolas Spiegelberg made changes -
          Field Original Value New Value
          Link This issue requires HBASE-3763 [ HBASE-3763 ]
          Nicolas Spiegelberg created issue -

            People

            • Assignee:
              Anoop Sam John
              Reporter:
              Nicolas Spiegelberg
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development