Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6581 Write to single replica in memory
  3. HDFS-7129

Metrics to track usage of memory for writes

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • HDFS-6581
    • 2.6.0
    • datanode
    • None
    • Reviewed

    Description

      A few metrics to evaluate feature usage and suggest improvements. Thanks to sureshms for some of these suggestions.

      1. Number of times a block in memory was read (before being ejected)
      2. Average block size for data written to memory tier
      3. Time the block was in memory before being ejected
      4. Number of blocks written to memory
      5. Number of memory writes requested but not satisfied (failed-over to disk)
      6. Number of blocks evicted without ever being read from memory
      7. Average delay between memory write and disk write (window where a node restart could cause data loss).
      8. Replicas written to disk by lazy writer
      9. Bytes written to disk by lazy writer
      10. Replicas deleted by application before being persisted to disk

      Attachments

        1. HDFS-7129.3.patch
          23 kB
          Xiaoyu Yao
        2. HDFS-7129.2.patch
          25 kB
          Xiaoyu Yao
        3. HDFS-7129.1.patch
          20 kB
          Xiaoyu Yao
        4. HDFS-7129.0.patch
          11 kB
          Xiaoyu Yao

        Issue Links

          Activity

            People

              xyao Xiaoyu Yao
              arp Arpit Agarwal
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: