Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8031 Follow-on work for erasure coding phase I (striping layout)
  3. HDFS-8901

Use ByteBuffer in striping positional read

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha2
    • 3.0.0-alpha2
    • erasure-coding
    • None
    • Reviewed

    Description

      Native erasure coder prefers to direct ByteBuffer for performance consideration. To prepare for it, this change uses ByteBuffer through the codes in implementing striping position read. It will also fix avoiding unnecessary data copying between striping read chunk buffers and decode input buffers.

      Attachments

        1. initial-poc.patch
          26 kB
          Kai Zheng
        2. HDFS-8901-v2.patch
          28 kB
          Kai Zheng
        3. HDFS-8901-v3.patch
          27 kB
          Kai Zheng
        4. HDFS-8901-v4.patch
          32 kB
          Kai Zheng
        5. HDFS-8901-v5.patch
          30 kB
          Kai Zheng
        6. HDFS-8901-v6.patch
          29 kB
          Kai Zheng
        7. HDFS-8901-v7.patch
          29 kB
          Kai Zheng
        8. HDFS-8901-v8.patch
          31 kB
          Li Bo
        9. HDFS-8901-v9.patch
          32 kB
          Li Bo
        10. HDFS-8901-v10.patch
          32 kB
          Kai Zheng
        11. HDFS-8901.v11.patch
          29 kB
          Youwei Wang
        12. HDFS-8901.v12.patch
          38 kB
          Youwei Wang
        13. HDFS-8901.v13.patch
          5 kB
          Youwei Wang
        14. HDFS-8901.v14.patch
          30 kB
          Youwei Wang
        15. HDFS-8901.v15.patch
          35 kB
          Sammi Chen
        16. HDFS-8901.v16.patch
          35 kB
          Sammi Chen
        17. HDFS-8901-v17.patch
          35 kB
          Sammi Chen
        18. HDFS-8901-v18.patch
          43 kB
          Sammi Chen
        19. HDFS-8901-v19.patch
          43 kB
          Sammi Chen
        20. HDFS-8901-v20.patch
          37 kB
          Sammi Chen

        Issue Links

          Is contained by

          Sub-task - The sub-task of the issue HDFS-8668 Erasure Coding: revisit buffer used for encoding and decoding.

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-8957 Consolidate client striping input stream codes for stateful read and positional read

          • Major - Major loss of function.
          • Patch Available
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-3246 pRead equivalent for direct read path

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. HDFS-14585 Backport HDFS-8901 Use ByteBuffer in DFSInputStream#read to branch2.9

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #998

          Web Link GitHub Pull Request #999

          Activity

            People

              Sammi Sammi Chen
              drankye Kai Zheng
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: