Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3697

Hadoop Counters API limits Oozie's working across different hadoop versions

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None

      Description

      Oozie uses Hadoop Counters API, by invoking Counters.getGroup(). However, in
      hadoop 23, org.apache.hadoop.mapred.Counters does not implement getGroup(). Its
      parent class AbstractCounters implements it. This is different from hadoop20X.
      As a result, Oozie compiled with either hadoop version does not work with the
      other version.
      A specific scenario, Oozie compiled with .23 and run against 205, does not
      update job status owing to a Counters API exception.

      Will explicit re-compilation against the relevant hadoop jars be required each
      time? This will prevent launching a uniform Oozie version across different
      clusters.

        Attachments

        1. MAPREDUCE-3697.patch
          31 kB
          Mahadev konar
        2. apicheck-hadoop-1.0.0-0.24.0-SNAPSHOT.sh
          1 kB
          Tom White
        3. MAPREDUCE-3697.patch
          28 kB
          Mahadev konar
        4. MAPREDUCE-3697.patch
          28 kB
          Mahadev konar
        5. MAPREDUCE-3697.patch
          23 kB
          Mahadev konar
        6. MAPREDUCE-3697-facadeV3.patch
          10 kB
          Alejandro Abdelnur
        7. MAPREDUCE-3697-facadeV2.patch
          6 kB
          Alejandro Abdelnur
        8. MAPREDUCE-3697-facade.patch
          6 kB
          Alejandro Abdelnur
        9. MAPREDUCE-3697.patch
          39 kB
          Mahadev konar
        10. MAPREDUCE-3697.patch
          39 kB
          Mahadev konar

          Issue Links

            Activity

              People

              • Assignee:
                mahadev Mahadev konar
                Reporter:
                johnvijoe John George
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: