Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.1
    • Component/s: hdfs-client, performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      BlockReader has been reimplemented to use direct byte buffers. If you use a custom socket factory, it must generate sockets that have associated Channels.

      Description

      BlockReader is currently quite complicated since it has to conform to the FSInputChecker inheritance structure. It would be much simpler to implement it standalone. Benchmarking indicates it's slightly faster, as well.

      1. hdfs-2129.txt
        63 kB
        Todd Lipcon
      2. hdfs-2129.txt
        63 kB
        Todd Lipcon
      3. hdfs-2129.txt
        63 kB
        Todd Lipcon
      4. hdfs-2129.txt
        52 kB
        Todd Lipcon
      5. hdfs-2129.txt
        48 kB
        Todd Lipcon
      6. hdfs-2129.txt
        44 kB
        Todd Lipcon
      7. seq-read-1gb-bench.png
        39 kB
        Todd Lipcon
      8. hdfs-2129-benchmark.png
        124 kB
        Todd Lipcon
      9. hdfs-2129.txt
        33 kB
        Todd Lipcon
      10. hdfs-2129.txt
        32 kB
        Todd Lipcon

        Issue Links

          Activity

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development