Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6581 Write to single replica in memory
  3. HDFS-7129

Metrics to track usage of memory for writes

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersConvert to IssueMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • HDFS-6581
    • 2.6.0
    • datanode
    • None
    • Reviewed

    Description

      A few metrics to evaluate feature usage and suggest improvements. Thanks to Suresh Srinivas for some of these suggestions.

      1. Number of times a block in memory was read (before being ejected)
      2. Average block size for data written to memory tier
      3. Time the block was in memory before being ejected
      4. Number of blocks written to memory
      5. Number of memory writes requested but not satisfied (failed-over to disk)
      6. Number of blocks evicted without ever being read from memory
      7. Average delay between memory write and disk write (window where a node restart could cause data loss).
      8. Replicas written to disk by lazy writer
      9. Bytes written to disk by lazy writer
      10. Replicas deleted by application before being persisted to disk

      Attachments

        1. HDFS-7129.0.patch
          11 kB
          Xiaoyu Yao
        2. HDFS-7129.1.patch
          20 kB
          Xiaoyu Yao
        3. HDFS-7129.2.patch
          25 kB
          Xiaoyu Yao
        4. HDFS-7129.3.patch
          23 kB
          Xiaoyu Yao

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            xyao Xiaoyu Yao Assign to me
            arp Arpit Agarwal
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment