Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3062

Need to capture the metrics for the network ios generate by dfs reads/writes and map/reduce shuffling and break them down by racks

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.0
    • Component/s: metrics
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Introduced additional log records for data transfers.

      Description

      In order to better understand the relationship between hadoop performance and the network bandwidth, we need to know
      what the aggregated traffic data in a cluster and its breakdown by racks. With these data, we can determine whether the network
      bandwidth is the bottleneck when certain jobs are running on a cluster.

        Attachments

        1. 3062-0.patch
          12 kB
          Chris Douglas
        2. 3062-1.patch
          17 kB
          Chris Douglas
        3. 3062-2.patch
          17 kB
          Chris Douglas
        4. 3062-3.patch
          21 kB
          Chris Douglas
        5. 3062-4.patch
          22 kB
          Chris Douglas
        6. 3062-5.patch
          22 kB
          Chris Douglas

          Issue Links

            Activity

              People

              • Assignee:
                chris.douglas Chris Douglas
                Reporter:
                runping Runping Qi
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: