Hadoop Common
  1. Hadoop Common
  2. HADOOP-2758

Reduce memory copies when data is read from DFS

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.17.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      DataNode takes 50% less CPU while serving data to clients.

      Description

      Currently datanode and client part of DFS perform multiple copies of data on the 'read path' (i.e. path from storage on datanode to user buffer on the client). This jira reduces these copies by enhancing data read protocol and implementation of read on both datanode and the client. I will describe the changes in next comment.

      Requirement is that this fix should reduce CPU used and should not cause regression in any benchmarks. It might not improve the benchmarks since most benchmarks are not cpu bound.

      1. HADOOP-2758.patch
        33 kB
        Raghu Angadi
      2. HADOOP-2758.patch
        33 kB
        Raghu Angadi
      3. HADOOP-2758.patch
        33 kB
        Raghu Angadi
      4. HADOOP-2758.patch
        33 kB
        Raghu Angadi
      5. HADOOP-2758.patch
        28 kB
        Raghu Angadi
      6. HADOOP-2758.patch
        21 kB
        Raghu Angadi

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Raghu Angadi
              Reporter:
              Raghu Angadi
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development