Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6389

Add support for LZF compression

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • io
    • None

    Description

      (note: related to HADOOP-4874)

      As per Doug's earlier comments, LZF does indeed look like a good compressor candidate for fast compression/decompression, good enough compression rate.
      From my testing it seems at least twice as fast at compression, and somewhat faster for decompressing than gzip.
      Code from http://h2database.googlecode.com/svn/trunk/h2/src/main/org/h2/compress/ is applicable, and I have tested it with json data.

      I hope to have more to spend on this in near future, but if someone else gets to this first that'd be good too.

      Attachments

        Issue Links

          is related to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-2404 native fast compression codec

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-3485 implement a Compressor based on LZF

          • Major - Major loss of function.
          • Closed

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-6349 Implement FastLZCodec for fastlz/lzo algorithm

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              cowtowncoder Tatu Saloranta
              Votes:
              4 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated: