Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11864

Document Metrics to track usage of memory for writes

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      HDFS-7129 introduced followings metrics which are not documented.

      	  // RamDisk metrics on read/write
      	  @Metric MutableCounterLong ramDiskBlocksWrite;
      	  @Metric MutableCounterLong ramDiskBlocksWriteFallback;
      	  @Metric MutableCounterLong ramDiskBytesWrite;
      	  @Metric MutableCounterLong ramDiskBlocksReadHits;
      	
      	  // RamDisk metrics on eviction
      	  @Metric MutableCounterLong ramDiskBlocksEvicted;
      	  @Metric MutableCounterLong ramDiskBlocksEvictedWithoutRead;
      	  @Metric MutableRate        ramDiskBlocksEvictionWindowMs;
      	  final MutableQuantiles[]   ramDiskBlocksEvictionWindowMsQuantiles;
      	
      	
      	  // RamDisk metrics on lazy persist
      	  @Metric MutableCounterLong ramDiskBlocksLazyPersisted;
      	  @Metric MutableCounterLong ramDiskBlocksDeletedBeforeLazyPersisted;
      	  @Metric MutableCounterLong ramDiskBytesLazyPersisted;
      	  @Metric MutableRate        ramDiskBlocksLazyPersistWindowMs;
      	  final MutableQuantiles[]   ramDiskBlocksLazyPersistWindowMsQuantiles;
      

        Issue Links

          Activity

          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - 2.8.1 became a security release. Moving fix-version to 2.8.2 after the fact.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Pushed to branch-2.8.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Pushed to branch-2.8 .
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Oh,Yes.. will cherry-pick to branch-2.8..thanks for reminding.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Oh,Yes.. will cherry-pick to branch-2.8 ..thanks for reminding.
          Hide
          linyiqun Yiqun Lin added a comment - - edited

          Thanks Brahma Reddy Battula for the review and commit. Why not merge this into branch-2.8 and add 2.8.2 into fix version? The versions 2.8.x will be fixed this issue after merging into branch-2.8.
          BTW, the patch can also apply cleanly into branch-2.8.

          Show
          linyiqun Yiqun Lin added a comment - - edited Thanks Brahma Reddy Battula for the review and commit. Why not merge this into branch-2.8 and add 2.8.2 into fix version? The versions 2.8.x will be fixed this issue after merging into branch-2.8. BTW, the patch can also apply cleanly into branch-2.8.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11770 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11770/)
          HDFS-11864. Document Metrics to track usage of memory for writes. (brahma: rev 52661e0912a79d1e851afc2b46c941ce952ca63f)

          • (edit) hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11770 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11770/ ) HDFS-11864 . Document Metrics to track usage of memory for writes. (brahma: rev 52661e0912a79d1e851afc2b46c941ce952ca63f) (edit) hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Committed to trunk,branch-2,branch-2.8.1 and branch-2.7..Patch cleanly applies all branches..Yiqun Lin for your contribution.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Committed to trunk , branch-2 , branch-2.8.1 and branch-2.7 ..Patch cleanly applies all branches.. Yiqun Lin for your contribution.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          +1, on trunk patch, will commit shortly.

          Show
          brahmareddy Brahma Reddy Battula added a comment - +1, on trunk patch, will commit shortly.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 12m 41s trunk passed
          +1 mvnsite 1m 15s trunk passed
          +1 mvnsite 0m 52s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 24s The patch does not generate ASF License warnings.
          15m 44s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HDFS-11864
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869425/HDFS-11864.001.patch
          Optional Tests asflicense mvnsite
          uname Linux 6e90f2a79553 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / d0f346a
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19559/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 12m 41s trunk passed +1 mvnsite 1m 15s trunk passed +1 mvnsite 0m 52s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 15m 44s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HDFS-11864 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12869425/HDFS-11864.001.patch Optional Tests asflicense mvnsite uname Linux 6e90f2a79553 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d0f346a modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HDFS-Build/19559/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Brahma Reddy Battula, the trunk, branch-2 and branch-2.8 are also missing these metrics in documenation, and should be updated, right? We would be better not only fixed this in branch-2.7.

          Yes, Need to update for all branches..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Brahma Reddy Battula, the trunk, branch-2 and branch-2.8 are also missing these metrics in documenation, and should be updated, right? We would be better not only fixed this in branch-2.7. Yes, Need to update for all branches..
          Hide
          linyiqun Yiqun Lin added a comment -

          Attach the initial patch. The metrics description based on the infos that given in HDFS-7129. Thanks for the review.
          Brahma Reddy Battula, the trunk, branch-2 and branch-2.8 are also missing these metrics in documenation, and should be updated, right? We would be better not only fixed this in branch-2.7.

          Show
          linyiqun Yiqun Lin added a comment - Attach the initial patch. The metrics description based on the infos that given in HDFS-7129 . Thanks for the review. Brahma Reddy Battula , the trunk, branch-2 and branch-2.8 are also missing these metrics in documenation, and should be updated, right? We would be better not only fixed this in branch-2.7.

            People

            • Assignee:
              linyiqun Yiqun Lin
              Reporter:
              brahmareddy Brahma Reddy Battula
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development