Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4562

Support for "FileSystemCounter" legacy counter group name for compatibility reasons is creating incorrect counter name

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 0.23.1, 2.0.0-alpha
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Hi Guys,
      I was investigating issue in Sqoop project(http://sqoop.apache.org/). Problem is that we are reporting number of written filesystem bytes back to the user and on Hadoop 0.23/2.x we're always getting 0. I've noticed that there was some refactorization in FileSystem counter related code and found MAPREDUCE-3542 requesting backward compatibility.

      Included patch seems to be adding counter "FileSystemCounter":

      hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/AbstractCounters.java:84
      legacyMap.put("FileSystemCounter", FileSystemCounter.class.getName());
      

      But it appears that original name is "FileSystemCounters" (Notice the plural "s" at the end of name):

      src/mapred/org/apache/hadoop/mapred/Task.java:91 (0.20.2)
      protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
      
      src/mapred/org/apache/hadoop/mapred/Task.java:109 (1.0.3)
      protected static final String FILESYSTEM_COUNTER_GROUP = "FileSystemCounters";
      

      I therefore believe that this counter should be renamed in order to provide backward compatibility. I might fix this discrepancy in Sqoop, but I believe that other projects/users might also be affected and therefore it would be better to fix it in upstream.

        Attachments

        1. MAPREDUCE-4562.patch
          3 kB
          Jarek Jarcec Cecho
        2. MAPREDUCE-4562.patch
          2 kB
          Jarek Jarcec Cecho

          Activity

            People

            • Assignee:
              jarcec Jarek Jarcec Cecho
              Reporter:
              jarcec Jarek Jarcec Cecho
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: