Hadoop Common
  1. Hadoop Common
  2. HADOOP-3164

Use FileChannel.transferTo() when data is read from DataNode.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.18.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Changed data node to use FileChannel.tranferTo() to transfer block data.
      Show
      Changed data node to use FileChannel.tranferTo() to transfer block data.

      Description

      HADOOP-2312 talks about using FileChannel's transferTo() and transferFrom() in DataNode.

      At the time DataNode neither used NIO sockets nor wrote large chunks of contiguous block data to socket. Hadoop 0.17 does both when data is seved to clients (and other datanodes). I am planning to try using transferTo() in the trunk. This might reduce DataNode's cpu by another 50% or more.

      Once HADOOP-1702 is committed, we can look into using transferFrom().

      1. HADOOP-3164.patch
        20 kB
        Raghu Angadi
      2. HADOOP-3164.patch
        17 kB
        Raghu Angadi
      3. HADOOP-3164.patch
        16 kB
        Raghu Angadi
      4. HADOOP-3164.patch
        15 kB
        Raghu Angadi
      5. HADOOP-3164.patch
        15 kB
        Raghu Angadi
      6. HADOOP-3164.patch
        14 kB
        Raghu Angadi

        Issue Links

          Activity

            People

            • Assignee:
              Raghu Angadi
              Reporter:
              Raghu Angadi
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development