Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11282

Document the missing metrics of DataNode Volume IO operations

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha2
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: documentation
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In HDFS-10959, it added many metrics of datanode volume io opearions. But it hasn't been documented. This JIRA addressed on this.

      1. metrics-rendered.png
        90 kB
        Arpit Agarwal
      2. HDFS-11282.004.patch
        4 kB
        Yiqun Lin
      3. HDFS-11282.003.patch
        4 kB
        Yiqun Lin
      4. HDFS-11282.002.patch
        4 kB
        Yiqun Lin
      5. HDFS-11282.001.patch
        3 kB
        Yiqun Lin

        Issue Links

          Activity

          Hide
          linyiqun Yiqun Lin added a comment -

          Attach a patch. Kindly review.

          Show
          linyiqun Yiqun Lin added a comment - Attach a patch. Kindly review.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 12m 50s trunk passed
          +1 mvnsite 0m 56s trunk passed
          +1 mvnsite 0m 49s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          15m 18s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11282
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845048/HDFS-11282.001.patch
          Optional Tests asflicense mvnsite
          uname Linux 21f750ebe98d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a4f6665
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17985/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 12m 50s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvnsite 0m 49s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 15m 18s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11282 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845048/HDFS-11282.001.patch Optional Tests asflicense mvnsite uname Linux 21f750ebe98d 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a4f6665 modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17985/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thank you for this improvement Yiqun Lin. Per-volume metrics are off by default. They can be enbabled by setting dfs.datanode.fileio.events.class to org.apache.hadoop.hdfs.server.datanode.ProfilingFileIoEvents.

          Show
          arpitagarwal Arpit Agarwal added a comment - Thank you for this improvement Yiqun Lin . Per-volume metrics are off by default. They can be enbabled by setting dfs.datanode.fileio.events.class to org.apache.hadoop.hdfs.server.datanode.ProfilingFileIoEvents .
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks Arpit Agarwal for the review and comments. New patch attached. In the latest patch, I addressed your comment and make some minor change as well to make the sentences more accurate. Kindly review.

          Show
          linyiqun Yiqun Lin added a comment - Thanks Arpit Agarwal for the review and comments. New patch attached. In the latest patch, I addressed your comment and make some minor change as well to make the sentences more accurate. Kindly review.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 12m 24s trunk passed
          +1 mvnsite 0m 53s trunk passed
          +1 mvnsite 0m 51s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          14m 49s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11282
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845147/HDFS-11282.002.patch
          Optional Tests asflicense mvnsite
          uname Linux e40267cd9c71 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c6a5b68
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17992/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 12m 24s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvnsite 0m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 14m 49s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11282 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845147/HDFS-11282.002.patch Optional Tests asflicense mvnsite uname Linux e40267cd9c71 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c6a5b68 modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17992/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment - - edited

          Thanks Yiqun Lin. Comments below.

          1. The rendered table looks wrong. The | need to be escaped. See
          2. Can you add that enabling per-volume metrics may have a performance impact?
          3. "Average Time" should be replaced with "Mean Time" in descriptions to be accurate.
          4. The description for the count metrics and the rate metric is the same e.g. TotalMetadataOperations and MetadataOperationRateNumOps. I am not sure that is correct.
          5. We can say that metadata operations include stat, list, mkdir, delete, move, open and posix_fadvise.
          6. The description Average time of file io error operations in milliseconds could be improved. It measures the mean time in milliseconds from the start of an operation to hitting a failure (assuming failures were seen on that volume).
          Show
          arpitagarwal Arpit Agarwal added a comment - - edited Thanks Yiqun Lin . Comments below. The rendered table looks wrong. The | need to be escaped. See Can you add that enabling per-volume metrics may have a performance impact? "Average Time" should be replaced with "Mean Time" in descriptions to be accurate. The description for the count metrics and the rate metric is the same e.g. TotalMetadataOperations and MetadataOperationRateNumOps. I am not sure that is correct. We can say that metadata operations include stat, list, mkdir, delete, move, open and posix_fadvise. The description Average time of file io error operations in milliseconds could be improved. It measures the mean time in milliseconds from the start of an operation to hitting a failure (assuming failures were seen on that volume).
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks for the patient review, Arpit Agarwal.

          The description for the count metrics and the rate metric is the same e.g. TotalMetadataOperations and MetadataOperationRateNumOps. I am not sure that is correct.

          I looked into this, the main difference between MutableCounter and MutableRate is that the value of the former is monotonically increasing and the latter will be reset after some interval time. Others comments have been addressed in the latest patch. New patch attached, please have a review. Thanks.

          Show
          linyiqun Yiqun Lin added a comment - Thanks for the patient review, Arpit Agarwal . The description for the count metrics and the rate metric is the same e.g. TotalMetadataOperations and MetadataOperationRateNumOps. I am not sure that is correct. I looked into this, the main difference between MutableCounter and MutableRate is that the value of the former is monotonically increasing and the latter will be reset after some interval time. Others comments have been addressed in the latest patch. New patch attached, please have a review. Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 29s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 15m 20s trunk passed
          +1 mvnsite 1m 0s trunk passed
          +1 mvnsite 0m 55s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 17s The patch does not generate ASF License warnings.
          18m 19s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11282
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845528/HDFS-11282.003.patch
          Optional Tests asflicense mvnsite
          uname Linux a97339cd1261 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e49e0a6
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18017/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 29s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 15m 20s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvnsite 0m 55s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 17s The patch does not generate ASF License warnings. 18m 19s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11282 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845528/HDFS-11282.003.patch Optional Tests asflicense mvnsite uname Linux a97339cd1261 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e49e0a6 modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18017/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Thanks Yiqun Lin. The patch looks good and the rendered site looks fine now.

          One nitpick: a interval should be an interval everywhere. Sorry I didn't catch that earlier.

          +1 otherwise.

          Show
          arpitagarwal Arpit Agarwal added a comment - Thanks Yiqun Lin . The patch looks good and the rendered site looks fine now. One nitpick: a interval should be an interval everywhere. Sorry I didn't catch that earlier. +1 otherwise.
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks Arpit Agarwal for the review and comment. New patch attached to address the comment.

          Show
          linyiqun Yiqun Lin added a comment - Thanks Arpit Agarwal for the review and comment. New patch attached to address the comment.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 14m 56s trunk passed
          +1 mvnsite 1m 2s trunk passed
          +1 mvnsite 0m 59s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          17m 50s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11282
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845916/HDFS-11282.004.patch
          Optional Tests asflicense mvnsite
          uname Linux abc52231830b 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 0b8a7c1
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18043/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 14m 56s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvnsite 0m 59s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 17m 50s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11282 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845916/HDFS-11282.004.patch Optional Tests asflicense mvnsite uname Linux abc52231830b 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0b8a7c1 modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18043/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          I committed this to trunk. Thanks for the contribution Yiqun Lin.

          Show
          arpitagarwal Arpit Agarwal added a comment - I committed this to trunk. Thanks for the contribution Yiqun Lin .
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11080 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11080/)
          HDFS-11282. Document the missing metrics of DataNode Volume IO (arp: rev 4a659ff40fca7c263d62ac7514afc100a4dbb1ed)

          • (edit) hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11080 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11080/ ) HDFS-11282 . Document the missing metrics of DataNode Volume IO (arp: rev 4a659ff40fca7c263d62ac7514afc100a4dbb1ed) (edit) hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
          Hide
          linyiqun Yiqun Lin added a comment -

          Thanks Arpit Agarwal.

          Show
          linyiqun Yiqun Lin added a comment - Thanks Arpit Agarwal .
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Cherry-picked to branch-2.

          Show
          arpitagarwal Arpit Agarwal added a comment - Cherry-picked to branch-2.

            People

            • Assignee:
              linyiqun Yiqun Lin
              Reporter:
              linyiqun Yiqun Lin
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development