HBase
  1. HBase
  2. HBASE-5521

Move compression/decompression to an encoder specific encoding context

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As part of working on HBASE-5313, we want to add a new columnar encoder/decoder. It makes sense to move compression to be part of encoder/decoder:
      1) a scanner for a columnar encoded block can do lazy decompression to a specific part of a key value object
      2) avoid an extra bytes copy from encoder to hblock-writer.

      If there is no encoder specified for a writer, the HBlock.Writer will use a default compression-context to do something very similar to today's code.

        Activity

        stack made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        stack made changes -
        Fix Version/s 0.98.0 [ 12323143 ]
        stack made changes -
        Fix Version/s 0.98.0 [ 12323143 ]
        stack made changes -
        Fix Version/s 0.95.0 [ 12324094 ]
        Fix Version/s 0.96.0 [ 12320040 ]
        Mikhail Bautin made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.10.patch [ 12518814 ]
        Ted Yu made changes -
        Fix Version/s 0.96.0 [ 12320040 ]
        Ted Yu made changes -
        Comment [ -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12517319/HBASE-5521.D2097.3.patch
          against trunk revision .

            +1 @author. The patch does not contain any @author tags.

            +1 tests included. The patch appears to include 12 new or modified tests.

            -1 javadoc. The javadoc tool appears to have generated -125 warning messages.

            +1 javac. The applied patch does not increase the total number of javac compiler warnings.

            -1 findbugs. The patch appears to introduce 158 new Findbugs (version 1.3.9) warnings.

            +1 release audit. The applied patch does not increase the total number of release audit warnings.

             -1 core tests. The patch failed these unit tests:
                               org.apache.hadoop.hbase.io.encoding.TestDataBlockEncoders

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/1122//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/1122//artifact/trunk/patchprocess/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/1122//console

        This message is automatically generated. ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.9.patch [ 12518781 ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.8.patch [ 12518757 ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.7.patch [ 12518742 ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.6.patch [ 12518539 ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.5.patch [ 12517499 ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.4.patch [ 12517375 ]
        Ted Yu made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hadoop Flags Reviewed [ 10343 ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.3.patch [ 12517319 ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.2.patch [ 12517294 ]
        Phabricator made changes -
        Attachment HBASE-5521.D2097.1.patch [ 12517118 ]
        He Yongqiang made changes -
        Assignee He Yongqiang [ he yongqiang ]
        He Yongqiang made changes -
        Attachment HBASE-5521.1.patch [ 12517115 ]
        He Yongqiang made changes -
        Attachment HBASE-5313-refactory.1.patch [ 12517114 ]
        He Yongqiang made changes -
        Field Original Value New Value
        Attachment HBASE-5313-refactory.1.patch [ 12517114 ]
        He Yongqiang created issue -

          People

          • Assignee:
            He Yongqiang
            Reporter:
            He Yongqiang
          • Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development