Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10047

Add a directbuffer Decompressor API to hadoop

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: io
    • Labels:
    • Release Note:
      Direct Bytebuffer decompressors for Zlib (Deflate & Gzip) and Snappy

      Description

      With the Zero-Copy reads in HDFS (HDFS-5260), it becomes important to perform all I/O operations without copying data into byte[] buffers or other buffers which wrap over them.

      This is a proposal for adding a DirectDecompressor interface to the io.compress, to indicate codecs which want to surface the direct buffer layer upwards.

      The implementation should work with direct heap/mmap buffers and cannot assume .array() availability.

        Attachments

        1. DirectCompressor.html
          13 kB
          Gopal Vijayaraghavan
        2. DirectDecompressor.html
          13 kB
          Gopal Vijayaraghavan
        3. HADOOP-10047-WIP.patch
          19 kB
          Gopal Vijayaraghavan
        4. HADOOP-10047-with-tests.patch
          22 kB
          Gopal Vijayaraghavan
        5. HADOOP-10047-redo-WIP.patch
          8 kB
          Gopal Vijayaraghavan
        6. HADOOP-10047-final.patch
          26 kB
          Gopal Vijayaraghavan
        7. HADOOP-10047-trunk.patch
          24 kB
          Gopal Vijayaraghavan
        8. decompress-benchmark.tgz
          2 kB
          Gopal Vijayaraghavan

          Issue Links

            Activity

              People

              • Assignee:
                gopalv Gopal Vijayaraghavan
                Reporter:
                gopalv Gopal Vijayaraghavan
              • Votes:
                0 Vote for this issue
                Watchers:
                16 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: