Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3697

Hadoop Counters API limits Oozie's working across different hadoop versions

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None

      Description

      Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in
      hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). Its
      parent class AbstractCounters implements it. This is different from hadoop20X.
      As a result, Oozie compiled with either hadoop version does not work with the
      other version.
      A specific scenario, Oozie compiled with .23 and run against 205, does not
      update job status owing to a Counters API exception.

      Will explicit re-compilation against the relevant hadoop jars be required each
      time? This will prevent launching a uniform Oozie version across different
      clusters.

      1. MAPREDUCE-3697.patch
        39 kB
        Mahadev konar
      2. MAPREDUCE-3697.patch
        39 kB
        Mahadev konar
      3. MAPREDUCE-3697-facade.patch
        6 kB
        Alejandro Abdelnur
      4. MAPREDUCE-3697-facadeV2.patch
        6 kB
        Alejandro Abdelnur
      5. MAPREDUCE-3697-facadeV3.patch
        10 kB
        Alejandro Abdelnur
      6. MAPREDUCE-3697.patch
        23 kB
        Mahadev konar
      7. MAPREDUCE-3697.patch
        28 kB
        Mahadev konar
      8. MAPREDUCE-3697.patch
        28 kB
        Mahadev konar
      9. apicheck-hadoop-1.0.0-0.24.0-SNAPSHOT.sh
        1 kB
        Tom White
      10. MAPREDUCE-3697.patch
        31 kB
        Mahadev konar

        Issue Links

          Activity

            People

            • Assignee:
              Mahadev konar
              Reporter:
              John George
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development