Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
DataNode takes 50% less CPU while serving data to clients.
Description
Currently datanode and client part of DFS perform multiple copies of data on the 'read path' (i.e. path from storage on datanode to user buffer on the client). This jira reduces these copies by enhancing data read protocol and implementation of read on both datanode and the client. I will describe the changes in next comment.
Requirement is that this fix should reduce CPU used and should not cause regression in any benchmarks. It might not improve the benchmarks since most benchmarks are not cpu bound.
Attachments
Attachments
Issue Links
- is depended upon by
-
HADOOP-1702 Reduce buffer copies when data is written to DFS
- Closed
- relates to
-
HADOOP-2154 Non-interleaved checksums would optimize block transfers.
- Resolved
-
HDFS-354 Data node process consumes 180% cpu
- Resolved