Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3164

Use FileChannel.transferTo() when data is read from DataNode.

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.18.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Changed data node to use FileChannel.tranferTo() to transfer block data.
      Show
      Changed data node to use FileChannel.tranferTo() to transfer block data.

      Description

      HADOOP-2312 talks about using FileChannel's transferTo() and transferFrom() in DataNode.

      At the time DataNode neither used NIO sockets nor wrote large chunks of contiguous block data to socket. Hadoop 0.17 does both when data is seved to clients (and other datanodes). I am planning to try using transferTo() in the trunk. This might reduce DataNode's cpu by another 50% or more.

      Once HADOOP-1702 is committed, we can look into using transferFrom().

        Attachments

        1. HADOOP-3164.patch
          14 kB
          Raghu Angadi
        2. HADOOP-3164.patch
          15 kB
          Raghu Angadi
        3. HADOOP-3164.patch
          15 kB
          Raghu Angadi
        4. HADOOP-3164.patch
          16 kB
          Raghu Angadi
        5. HADOOP-3164.patch
          17 kB
          Raghu Angadi
        6. HADOOP-3164.patch
          20 kB
          Raghu Angadi

        Issue Links

          Activity

            People

            • Assignee:
              rangadi Raghu Angadi
              Reporter:
              rangadi Raghu Angadi

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment