Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-2834

ByteBuffer-based read API for DFSInputStream

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.2-alpha
    • Component/s: hdfs-client, performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The DFSInputStream read-path always copies bytes into a JVM-allocated byte[]. Although for many clients this is desired behaviour, in certain situations, such as native-reads through libhdfs, this imposes an extra copy penalty since the byte[] needs to be copied out again into a natively readable memory area.

      For these cases, it would be preferable to allow the client to supply its own buffer, wrapped in a ByteBuffer, to avoid that final copy overhead.

      1. HDFS-2834.10.patch
        48 kB
        Henry Robinson
      2. HDFS-2834.11.patch
        60 kB
        Henry Robinson
      3. HDFS-2834.3.patch
        63 kB
        Henry Robinson
      4. HDFS-2834.4.patch
        58 kB
        Henry Robinson
      5. HDFS-2834.5.patch
        59 kB
        Henry Robinson
      6. HDFS-2834.6.patch
        59 kB
        Henry Robinson
      7. HDFS-2834.7.patch
        61 kB
        Henry Robinson
      8. HDFS-2834.8.patch
        61 kB
        Henry Robinson
      9. HDFS-2834.9.patch
        49 kB
        Henry Robinson
      10. HDFS-2834.patch
        64 kB
        Henry Robinson
      11. HDFS-2834.patch
        66 kB
        Henry Robinson
      12. hdfs-2834-libhdfs-benchmark.png
        43 kB
        Henry Robinson
      13. HDFS-2834-no-common.patch
        60 kB
        Henry Robinson

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Henry Robinson
              Reporter:
              Henry Robinson
            • Votes:
              1 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development