HBase
  1. HBase
  2. HBASE-5470

Make DataBlockEncodingTool work correctly with no native compression codecs loaded

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.0
    • Component/s: None
    • Labels:
      None

      Description

      DataBlockEncodingTool was fixed as part of porting data block encoding (HBASE-4218) to 89-fb (https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1245291, https://reviews.facebook.net/D1659). The bug appeared when using GZ as baseline compression codec but not loading native Hadoop libraries, in which case the compressor instance would be null. The purpose of this JIRA is to bring the trunk version of DataBlockEncodingTool to parity with the 89-fb version, and further improvements to the tool will be made separately.

        Activity

        Hide
        Phabricator added a comment -

        mbautin requested code review of "[jira] HBASE-5470 HBase
        HBASE-5470

        Make DataBlockEncodingTool work correctly with no native compression codecs loaded".
        Reviewers: JIRA, Kannan, mcorgan, lhofhansl, todd, stack

        DataBlockEncodingTool was fixed as part of porting data block encoding (HBASE-4218) to 89-fb (https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1245291, https://reviews.facebook.net/D1659). The bug being fixed here appeared when using GZ as baseline compression codec but not loading native Hadoop libraries, in which case the compressor instance would be null.

        TEST PLAN
        Run DataBlockEncoding tool with GZ (no native codecs) and LZO (with native codecs) as baseline (Hadoop-level) compression codecs

        REVISION DETAIL
        https://reviews.facebook.net/D1917

        AFFECTED FILES
        src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java

        MANAGE HERALD DIFFERENTIAL RULES
        https://reviews.facebook.net/herald/view/differential/

        WHY DID I GET THIS EMAIL?
        https://reviews.facebook.net/herald/transcript/4071/

        Tip: use the X-Herald-Rules header to filter Herald messages in your client.

        Show
        Phabricator added a comment - mbautin requested code review of " [jira] HBASE-5470 HBase HBASE-5470 Make DataBlockEncodingTool work correctly with no native compression codecs loaded". Reviewers: JIRA, Kannan, mcorgan, lhofhansl, todd, stack DataBlockEncodingTool was fixed as part of porting data block encoding ( HBASE-4218 ) to 89-fb ( https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1245291 , https://reviews.facebook.net/D1659 ). The bug being fixed here appeared when using GZ as baseline compression codec but not loading native Hadoop libraries, in which case the compressor instance would be null. TEST PLAN Run DataBlockEncoding tool with GZ (no native codecs) and LZO (with native codecs) as baseline (Hadoop-level) compression codecs REVISION DETAIL https://reviews.facebook.net/D1917 AFFECTED FILES src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java MANAGE HERALD DIFFERENTIAL RULES https://reviews.facebook.net/herald/view/differential/ WHY DID I GET THIS EMAIL? https://reviews.facebook.net/herald/transcript/4071/ Tip: use the X-Herald-Rules header to filter Herald messages in your client.
        Hide
        Phabricator added a comment -

        tedyu has accepted the revision "[jira] HBASE-5470 HBase
        HBASE-5470

        Make DataBlockEncodingTool work correctly with no native compression codecs loaded".

        REVISION DETAIL
        https://reviews.facebook.net/D1917

        BRANCH
        improve_data_blk_enc_tool

        Show
        Phabricator added a comment - tedyu has accepted the revision " [jira] HBASE-5470 HBase HBASE-5470 Make DataBlockEncodingTool work correctly with no native compression codecs loaded". REVISION DETAIL https://reviews.facebook.net/D1917 BRANCH improve_data_blk_enc_tool
        Hide
        Phabricator added a comment -

        mbautin has committed the revision "[jira] HBASE-5470 Make DataBlockEncodingTool work correctly with no native compression codecs loaded".

        REVISION DETAIL
        https://reviews.facebook.net/D1917

        COMMIT
        https://reviews.facebook.net/rHBASE1293057

        Show
        Phabricator added a comment - mbautin has committed the revision " [jira] HBASE-5470 Make DataBlockEncodingTool work correctly with no native compression codecs loaded". REVISION DETAIL https://reviews.facebook.net/D1917 COMMIT https://reviews.facebook.net/rHBASE1293057
        Hide
        stack added a comment -

        Resolving as Mikhail committed the issue.

        Show
        stack added a comment - Resolving as Mikhail committed the issue.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-security #121 (See https://builds.apache.org/job/HBase-TRUNK-security/121/)
        [jira] HBASE-5470 Make DataBlockEncodingTool work correctly with no native
        compression codecs loaded

        Summary:
        DataBlockEncodingTool was fixed as part of porting data block encoding
        (HBASE-4218) to 89-fb
        (https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1245291,
        https://reviews.facebook.net/D1659). The bug being fixed here appeared when
        using GZ as baseline compression codec but not loading native Hadoop libraries,
        in which case the compressor instance would be null.

        Test Plan:
        Run DataBlockEncoding tool with GZ (no native codecs) and LZO (with native
        codecs) as baseline (Hadoop-level) compression codecs

        Reviewers: JIRA, Kannan, mcorgan, lhofhansl, todd, stack, tedyu

        Reviewed By: tedyu

        Differential Revision: https://reviews.facebook.net/D1917 (Revision 1293057)

        Result = SUCCESS
        mbautin :
        Files :

        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-security #121 (See https://builds.apache.org/job/HBase-TRUNK-security/121/ ) [jira] HBASE-5470 Make DataBlockEncodingTool work correctly with no native compression codecs loaded Summary: DataBlockEncodingTool was fixed as part of porting data block encoding ( HBASE-4218 ) to 89-fb ( https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1245291 , https://reviews.facebook.net/D1659 ). The bug being fixed here appeared when using GZ as baseline compression codec but not loading native Hadoop libraries, in which case the compressor instance would be null. Test Plan: Run DataBlockEncoding tool with GZ (no native codecs) and LZO (with native codecs) as baseline (Hadoop-level) compression codecs Reviewers: JIRA, Kannan, mcorgan, lhofhansl, todd, stack, tedyu Reviewed By: tedyu Differential Revision: https://reviews.facebook.net/D1917 (Revision 1293057) Result = SUCCESS mbautin : Files : /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #2669 (See https://builds.apache.org/job/HBase-TRUNK/2669/)
        [jira] HBASE-5470 Make DataBlockEncodingTool work correctly with no native
        compression codecs loaded

        Summary:
        DataBlockEncodingTool was fixed as part of porting data block encoding
        (HBASE-4218) to 89-fb
        (https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1245291,
        https://reviews.facebook.net/D1659). The bug being fixed here appeared when
        using GZ as baseline compression codec but not loading native Hadoop libraries,
        in which case the compressor instance would be null.

        Test Plan:
        Run DataBlockEncoding tool with GZ (no native codecs) and LZO (with native
        codecs) as baseline (Hadoop-level) compression codecs

        Reviewers: JIRA, Kannan, mcorgan, lhofhansl, todd, stack, tedyu

        Reviewed By: tedyu

        Differential Revision: https://reviews.facebook.net/D1917 (Revision 1293057)

        Result = SUCCESS
        mbautin :
        Files :

        • /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #2669 (See https://builds.apache.org/job/HBase-TRUNK/2669/ ) [jira] HBASE-5470 Make DataBlockEncodingTool work correctly with no native compression codecs loaded Summary: DataBlockEncodingTool was fixed as part of porting data block encoding ( HBASE-4218 ) to 89-fb ( https://reviews.facebook.net/rHBASEEIGHTNINEFBBRANCH1245291 , https://reviews.facebook.net/D1659 ). The bug being fixed here appeared when using GZ as baseline compression codec but not loading native Hadoop libraries, in which case the compressor instance would be null. Test Plan: Run DataBlockEncoding tool with GZ (no native codecs) and LZO (with native codecs) as baseline (Hadoop-level) compression codecs Reviewers: JIRA, Kannan, mcorgan, lhofhansl, todd, stack, tedyu Reviewed By: tedyu Differential Revision: https://reviews.facebook.net/D1917 (Revision 1293057) Result = SUCCESS mbautin : Files : /hbase/trunk/src/test/java/org/apache/hadoop/hbase/regionserver/DataBlockEncodingTool.java

          People

          • Assignee:
            Mikhail Bautin
            Reporter:
            Mikhail Bautin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development