Issue Details (XML | Word | Printable)

Key: HADOOP-3164
Type: Improvement Improvement
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: Raghu Angadi
Reporter: Raghu Angadi
Votes: 0
Watchers: 4
Operations

If you were logged in you would be able to see more operations.
Hadoop Common

Use FileChannel.transferTo() when data is read from DataNode.

Created: 03/Apr/08 07:22 PM   Updated: 08/Jul/09 04:43 PM
Return to search
Component/s: None
Affects Version/s: None
Fix Version/s: 0.18.0

Time Tracking:
Not Specified

File Attachments:
  Size
Text File Licensed for inclusion in ASF works HADOOP-3164.patch 2008-04-23 10:32 PM Raghu Angadi 20 kB
Text File Licensed for inclusion in ASF works HADOOP-3164.patch 2008-04-21 07:45 PM Raghu Angadi 17 kB
Text File Licensed for inclusion in ASF works HADOOP-3164.patch 2008-04-18 09:53 PM Raghu Angadi 16 kB
Text File Licensed for inclusion in ASF works HADOOP-3164.patch 2008-04-08 03:44 AM Raghu Angadi 15 kB
Text File Licensed for inclusion in ASF works HADOOP-3164.patch 2008-04-08 03:29 AM Raghu Angadi 15 kB
Text File Licensed for inclusion in ASF works HADOOP-3164.patch 2008-04-05 12:30 AM Raghu Angadi 14 kB
Issue Links:
Reference
 

Hadoop Flags: Reviewed
Release Note:
Changed data node to use FileChannel.tranferTo() to transfer block data.
Resolution Date: 25/Apr/08 12:10 AM


 Description  « Hide
HADOOP-2312 talks about using FileChannel's transferTo() and transferFrom() in DataNode.

At the time DataNode neither used NIO sockets nor wrote large chunks of contiguous block data to socket. Hadoop 0.17 does both when data is seved to clients (and other datanodes). I am planning to try using transferTo() in the trunk. This might reduce DataNode's cpu by another 50% or more.

Once HADOOP-1702 is committed, we can look into using transferFrom().



 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order