Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12332

Logging improvement suggestion for SampleStat function MinMax.add

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • 3.0.0-alpha4
    • None
    • metrics
    • None

    Description

      In order to debug performance anomlies, it is better to see when the max and min value get updated in metrics.
      For example, in our workload we would like to see the longest latency of a write block request on DataNode.

      The following metric updating code could possibly update the min/max value in SampleStat class:

          datanode.getMetrics().addWriteBlockOp(elapsed());
          datanode.getMetrics().incrWritesFromClient(peer.isLocal(), size);
      

      Here I am attaching an attempt to patch it.
      The patch just adds additional DEBUG level logging in the MinMax class and prints the new value when it gets updated.

      Please let me know what you think. For example, if this patch would introduce too much noises, what is the best way to trace the read/writeBlock that has the minimum/maximum latency?
      Any comments are appreciated!

      Attachments

        1. hdfs-12332.patch
          1 kB
          Xu Zhao

        Activity

          People

            Unassigned Unassigned
            xzhao9 Xu Zhao
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated: