Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-7206

Integrate Snappy compression

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.21.0
    • 1.0.2, 0.23.0
    • None
    • None
    • Reviewed

    Description

      Google release Zippy as an open source (APLv2) project called Snappy (http://code.google.com/p/snappy). This tracks integrating it into Hadoop.

      Snappy is a compression/decompression library. It does not aim for maximum compression, or compatibility with any other compression library; instead, it aims for very high speeds and reasonable compression. For instance, compared to the fastest mode of zlib, Snappy is an order of magnitude faster for most inputs, but the resulting compressed files are anywhere from 20% to 100% bigger. On a single core of a Core i7 processor in 64-bit mode, Snappy compresses at about 250 MB/sec or more and decompresses at about 500 MB/sec or more.

      Attachments

        1. HADOOP-7206-20120302.txt
          50 kB
          Vinod Kumar Vavilapalli
        2. HADOOP-7206-20120223.txt
          50 kB
          Vinod Kumar Vavilapalli
        3. HADOOP-7206new-c.patch
          52 kB
          Alejandro Abdelnur
        4. HADOOP-7206new-b.patch
          52 kB
          Alejandro Abdelnur
        5. HADOOP-7206new.patch
          45 kB
          Alejandro Abdelnur
        6. HADOOP-7206revertplusnew-b.patch
          51 kB
          Alejandro Abdelnur
        7. HADOOP-7206revertplusnew.patch
          49 kB
          Alejandro Abdelnur
        8. v5-HADOOP-7206-snappy-codec-using-snappy-java.txt
          18 kB
          T Jake Luciani
        9. v4-HADOOP-7206-snappy-codec-using-snappy-java.txt
          18 kB
          T Jake Luciani
        10. v3-HADOOP-7206-snappy-codec-using-snappy-java.txt
          19 kB
          T Jake Luciani
        11. v2-HADOOP-7206-snappy-codec-using-snappy-java.txt
          19 kB
          T Jake Luciani
        12. HADOOP-7206-002.patch
          38 kB
          issei yoshida
        13. HADOOP-7206.patch
          38 kB
          issei yoshida

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-7990 TFile to allow Snappy compression

          • Major - Major loss of function.
          • Open
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-7408 Add javadoc for SnappyCodec

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-7407 Snappy integration breaks HDFS build.

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Task - A task that needs to be done. HBASE-3691 Add compressor support for 'snappy', google's compressor

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              tucu00 Alejandro Abdelnur
              eli Eli Collins
              Votes:
              13 Vote for this issue
              Watchers:
              52 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: