Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-15507

Add MapReduce counters about EC bytes read

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.0
    • None
    • None
    • Reviewed

    Description

      HDFS has added Erasure Coding support in HDFS-7285. There are HDFS level ReadStatistics so from DFSClient we can know how much reads are EC/replication.

      In order for users to have a better view of how much of their workload is impacted by EC, we can expose EC read bytes to File System Counters, and to MapReduce's job counters. This way, end users can tell from MR jobs directly.

      Attachments

        1. image-2018-05-31-15-29-45-729.png
          174 kB
          Xiao Chen
        2. HADOOP-15507.01.patch
          13 kB
          Xiao Chen

        Activity

          People

            xiaochen Xiao Chen
            xiaochen Xiao Chen
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: