Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8031 Follow-on work for erasure coding phase I (striping layout)
  3. HDFS-7674

[umbrella] Adding metrics for Erasure Coding

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Done
    • None
    • None
    • None
    • None

    Description

      As the design (in HDFS-7285) indicates, erasure coding involves non-trivial impact and workload for NameNode, DataNode and client; it also allows configurable and pluggable erasure codec and schema with flexible tradeoff options (see HDFS-7337). To support necessary analysis and adjustment, we'd better have various meaningful metrics for the EC support, like encoding/decoding tasks, recovered blocks, read/transferred data size, computation time and etc.

      Attachments

        Issue Links

          contains

          Sub-task - The sub-task of the issue HDFS-8410 Add computation time metrics to datanode for ECWorker

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-8411 Add bytes count metrics to datanode for ECWorker

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-8449 Add tasks count metrics to datanode for ECWorker

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-8529 Add blocks count metrics to datanode for ECWorker

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-8672 Erasure Coding: Add EC-related Metrics to NN (seperate striped blocks count from UnderReplicatedBlocks count)

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-8196 Post enabled Erasure Coding Policies on NameNode UI

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-11371 Document missing metrics of erasure coding

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              libo-intel Li Bo
              drankye Kai Zheng
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: