Hadoop Common
  1. Hadoop Common
  2. HADOOP-3062

Need to capture the metrics for the network ios generate by dfs reads/writes and map/reduce shuffling and break them down by racks

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.0
    • Component/s: metrics
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Introduced additional log records for data transfers.

      Description

      In order to better understand the relationship between hadoop performance and the network bandwidth, we need to know
      what the aggregated traffic data in a cluster and its breakdown by racks. With these data, we can determine whether the network
      bandwidth is the bottleneck when certain jobs are running on a cluster.

      1. 3062-0.patch
        12 kB
        Chris Douglas
      2. 3062-1.patch
        17 kB
        Chris Douglas
      3. 3062-2.patch
        17 kB
        Chris Douglas
      4. 3062-3.patch
        21 kB
        Chris Douglas
      5. 3062-4.patch
        22 kB
        Chris Douglas
      6. 3062-5.patch
        22 kB
        Chris Douglas

        Issue Links

          Activity

            People

            • Assignee:
              Chris Douglas
              Reporter:
              Runping Qi
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development