Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-15666

add average latency information to the SlowPeerReport

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: hdfs
    • Labels:
      None

      Description

      In namenode's jmx, there is a SlowDisksReport like this:

      [{"SlowDiskID":"dn3:disk1","Latencies":{"WRITE":1000.1}},{"SlowDiskID":"dn2:disk2","Latencies":{"WRITE":1000.1}},{"SlowDiskID":"dn1:disk2","Latencies":{"READ":1000.3}},{"SlowDiskID":"dn1:disk1","Latencies":{"METADATA":1000.1,"READ":1000.8}}]
      

      So we can know the disk io letency from this report.However, SlowPeersReport dosen't have average latency:

      [{"SlowNode":"node4","ReportingNodes":["node1"]},{"SlowNode":"node2","ReportingNodes":["node1","node3"]},{"SlowNode":"node1","ReportingNodes":["node2"]}]
      

      I think we should add the average latency to the report, which can get from org.apache.hadoop.hdfs.server.protocol.SlowPeerReports#slowPeers.

      After adding the average latency, the SlowPeerReport can be like this:

      [{"SlowNode":"node4","ReportingNodes":[{"nodeId":"node1","averageLatency":2000.0}]},{"SlowNode":"node2","ReportingNodes":[{"nodeId":"node1","averageLatency":2000.0},{"nodeId":"node3","averageLatency":1000.0}]},{"SlowNode":"node1","ReportingNodes":[{"nodeId":"node2","averageLatency":2000.0}]}]

        Attachments

        1. HDFS-15666.001.patch
          15 kB
          Haibin Huang
        2. HDFS-15666.002.patch
          15 kB
          Haibin Huang
        3. HDFS-15666-003.patch
          16 kB
          Haibin Huang
        4. HDFS-15666-004.patch
          16 kB
          Haibin Huang

          Activity

            People

            • Assignee:
              huanghaibin Haibin Huang
              Reporter:
              huanghaibin Haibin Huang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: