Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3062

Need to capture the metrics for the network ios generate by dfs reads/writes and map/reduce shuffling and break them down by racks

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.19.0
    • metrics
    • None
    • Incompatible change, Reviewed
    • Introduced additional log records for data transfers.

    Description

      In order to better understand the relationship between hadoop performance and the network bandwidth, we need to know
      what the aggregated traffic data in a cluster and its breakdown by racks. With these data, we can determine whether the network
      bandwidth is the bottleneck when certain jobs are running on a cluster.

      Attachments

        1. 3062-0.patch
          12 kB
          Christopher Douglas
        2. 3062-1.patch
          17 kB
          Christopher Douglas
        3. 3062-2.patch
          17 kB
          Christopher Douglas
        4. 3062-3.patch
          21 kB
          Christopher Douglas
        5. 3062-4.patch
          22 kB
          Christopher Douglas
        6. 3062-5.patch
          22 kB
          Christopher Douglas

        Issue Links

          incorporates

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-3658 Incorrect destination IP logged for receiving blocks

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Closed

          Activity

            People

              cdouglas Christopher Douglas
              runping Runping Qi
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: