Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3164

Use FileChannel.transferTo() when data is read from DataNode.

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.18.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Changed data node to use FileChannel.tranferTo() to transfer block data.
      Show
      Changed data node to use FileChannel.tranferTo() to transfer block data.

      Description

      HADOOP-2312 talks about using FileChannel's transferTo() and transferFrom() in DataNode.

      At the time DataNode neither used NIO sockets nor wrote large chunks of contiguous block data to socket. Hadoop 0.17 does both when data is seved to clients (and other datanodes). I am planning to try using transferTo() in the trunk. This might reduce DataNode's cpu by another 50% or more.

      Once HADOOP-1702 is committed, we can look into using transferFrom().

        Attachments

        1. HADOOP-3164.patch
          20 kB
          Raghu Angadi
        2. HADOOP-3164.patch
          17 kB
          Raghu Angadi
        3. HADOOP-3164.patch
          16 kB
          Raghu Angadi
        4. HADOOP-3164.patch
          15 kB
          Raghu Angadi
        5. HADOOP-3164.patch
          15 kB
          Raghu Angadi
        6. HADOOP-3164.patch
          14 kB
          Raghu Angadi

          Issue Links

            Activity

              People

              • Assignee:
                rangadi Raghu Angadi
                Reporter:
                rangadi Raghu Angadi
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: