Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-1323

Pool/share file channels for HDFS read

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: datanode, performance
    • Labels:
      None

      Description

      Currently, all reads in HDFS require opening and closing the underlying block/meta filechannels. We could pool these filechannels and save some system calls and other work. Since HDFS read requests can be satisfied by positioned reads and transferTos, we can even share these filechannels between concurrently executing requests.

      The attached patch was benchmarked as part of work on HDFS-918 and exhibited a 10% performance increase for small random reads.

      This does not affect client logic and involves minimal change to server logic. Patch is based on branch 20-append.

        Attachments

        1. hdfs-1323-trunk.txt
          32 kB
          Todd Lipcon
        2. hdfs-1323-20100730.patch
          39 kB
          Jay Booth

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jbooth Jay Booth
              • Votes:
                0 Vote for this issue
                Watchers:
                18 Start watching this issue

                Dates

                • Created:
                  Updated: